RNNs perform task computations by dynamically warping neural representations
Authors: Arthur Pellegrino
Angus Chadwick
Venue: NeurIPS 2025 poster ()
Keywords: RNN
Computational Neuroscience
Dynamical Systems
Neural Representations
Primary area: neuroscience_and_cognitive_science
TL;DR: RNNs used in computational neuroscience lie on manifolds whose geometry provides insights into their computations.

Abstract: Analysing how neural networks represent data features in their activations can help interpret how they perform tasks. Hence, a long line of work has focused on mathematically characterising the geometry of such "neural representations." In parallel, machine learning has seen a surge of interest in understanding how dynamical systems perform computations on time-varying input data. Yet, the link between computation-through-dynamics and representational geometry remains poorly understood. Here, we hypothesise that recurrent neural networks (RNNs) perform computations by dynamically warping their representations of task variables. To test this hypothesis, we develop a Riemannian geometric framework that enables the derivation of the manifold topology and geometry of a dynamical system from the manifold of its inputs. By characterising the time-varying geometry of RNNs, we show that dynamic warping is a fundamental feature of their computations.

RNNs perform task computations by dynamically warping neural representations
Arthur Pellegrino The Gatsby Unit University College London pellegrino@ucl ac. uk
Angus Chadwick School of Informatics University of Edinburgh angus chadwi ckéed. ac uk
Abstract
Analysing how neural networks represent data features in their activations help interpret how they perform tasks. Hence, a long line of work has focused on mathe- matically characterising the geometry of such "neural representations." In parallel, machine learning has seen a surge of interest understanding how dynamical systems perform computations on time-varying input data. Yet, the link between computation-througe wawiiionnttoouug-yymmimiii and representational geometry remains poorly un- derstood. Here, hypothesise that recurrent neural networks (RNNs) perform computations by dynamically warping their representations of task variables. To this hypothesis, we develop Riemannian geometric framework that enables the derivation of the manifold topology and geometry of dynamical system from the manifold of its inputs. By characterising the mareiia geometry of RNNs, we show that dynamic warping is fundamental feature of their computations.
Introduction
Understanding the low-dimensional neural representations used by biological and artificial neural networks to solve complex tasks is key challenge in machine learning. 1.2 For example, low-dimensional activity manifolds have been ob- served in neural systems performing vision or language tasks. More generally, data from the natural world often lie on low-dimensional manifolds. and the computations performed by neural networks trained on these data can be interpreted as transformations of these data manifolds into neural activation manifolds.
((x.u()
Manilold of inpuls
Dynamical system
PNN, bft
Hepre- sentation
However, existing work on the geometry neural represen- tations has focused the static, time-independent, setting." In contrast, the geometry of the dynamic representations formed by neural networks driven by time-varying inputs remains largely unexplored.
Figure 1: Dynamical systems receiv- ing inputs on low-dimensional man- ifold of functions constrained to -ow-dimensional manifold of states. The Riemannian geometry of this mani- fold can provide insights into dynami- cal representations and computations.
Dynamical system models, described as differential equa- tions, are ubiquitous in science and machine learning. The development and application of methods infer dynamical systems from 3-10 or train dynamical systems to solve tasks 11 are now central to computational neuroscience, 12-14 physics," 15 chemistry 16 and genomics. 17.18 In machine learning, diffusion mod- els based on stochastic differential equations achieve state-of-the-art performance for generative modelling of images." In computational neuroscience, recurrent neural networks (RNNs) defined using differential equations are used routinely to investigate neural computations underlying various behavioural tasks.".
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

However, both data-driven and mask-optimised nonlinear dynamical systems models are typically viewed as "black boxes". with their behaviour lacking a clear and interpretable explanation. 21-23 Attempts to interpret dynamical systems models typically focus on the topology and geometry of fixed point manifolds, and the locally linearised dynamics about them. 21 This approach has provided insight into the computations used by RNNs to solve a range of tasks. 24 Nevertheless, the study of dynamics near fixed points faces a number of limitations. First, many systems exhibit complex, time-varying activity patterns which do not settle into fixed point, 25,26 or nonlinear dynamics before settling into a fixed point which are crucial to the computation. 27-29 Second, the characterisation of fixed point manifolds is insufficient to determine the geometry and topology of the full activity manifold, and therefore provides an incomplete understanding of the system.
Thus, two key questions remain. First, what is the relationship between the geometry and topology of the data manifold and that of the dynamical system's is activity, and how does this relate to the internal dynamics of the system? Second, in the case where the dynamical system is trained to perform specific task, as are RNNs, how can these geometric and topological properties be related to the computations performed by the network to solve the task?
Here, we show that Riemannian geometry can be leveraged understand the neural representations used by dynamical systems to perform time-dependent task computations. Specifically, we prove that under weak constraints the dynamical system's 's state is restricted to a manifold whose topology is closely related to that of the input data manifold. Since the input manifold is independent of the trained parameters of the model, the geometry of the neural activity manifold reflects the computation it performs on these inputs. To analyse this geometry, we derive the pullback Ri Riemannian metric on the neural manifold of a general class of dynamical systems. This metrie naturally describes the local encoding of task variables and is defined via an adjoint differential equation. Using this metric we demonstrate that several classic RNN models from neuroscience solve tasks by dynamically warping their internal representations of input variables. Overall, we offer novel formal mathematical framework to characterise the geometry of dynamical systems beyond attractor manifolds, which can be used to study RNN representations and computations.
Contributions
We derive relationship between the topology of the manifold of inputs to a dynamical system and the manifold over which the state of the system is constrained. We prove that for inputs on m-dimennional manifold, the state of the system is constrained to an m + 1-dimensional manifold.
We derive the pullback of the metric, from the neural state-space to the input manifold for a general class of dynamical systems. In several examples show that this metric is reflection of the computations performed by the dynamical system on its inputs to solve a task.
In an RNN trained on a contextual decision-making task we show that, over time, the manifold becomes warped so as to compress irrelevant input information. In a sequential working memory task, we show that RNN activity lies on a hyper-torus, whose extrinsic and intrinsic geometry dynamically warps to retrieve different memories at different points in time.
Related works
Attractor manifolds and dynamical systems interpretability. Attractor manifolds of dynamical system models have been widely studied to interpret the underlying computations of RNNs, 30.31 denoising diffusion models 32,38 and neuralODEs** 34 in tasks spanning language reinforcement learn- ing" and computational neuroscience." 21,36 Here instead, we show that the computations performed by a dynamical system are reflected by the geometry of the manifold over which its state lies. Works beside ours have suggested going beyond the study of fixed points, for example via distillation of nonlinear dynamics to low-dimensional subspace." 37 or by applying dimensionality reduction meth- ods to the systems' dynamics23.8 or trajectories. 39.40 In contrast, we show that the low-dimensional geometry of the system can be exactly mathematically derived via an adjoint dynamical system.
Riemannian neural representations. The Riemannian geometry of neural activity in deep networks has been extensively studied, 7,41-43 7.41-4 This has helped interpret how neural networks represent features of images, 44,45 RL task " and t7x in their neural activity. However, existing frameworks typically assume static inputs. Here, we mathematically characterise the dynamic geometry of neural networks described via differential equations receiving time-varying inputs.
2

---

2 Riemannian geometry can link representation and computation
Review of Riemannian geometry. Before studying the manifolds of dynamical systems, first consider a static deep neural network to provide insight into a simplified setting. Throughout, we data manifold M to ambient space IR..
work with function yp : .M- In the case where yp is a deep neural network, this ambient space may be that of the n neurons in one of its hidden layers. In the next section, we generalise this setting to the time- varying case, where yp is the solution to a dynamical system, and IR" its state-space.
R mapping inputs
Following previous work, 7.41,42 we can derive a metric on M. called the pullback metric, which captures the change in geometry generated by the neural network sp We denote the tangent space at point p E .M as T.M. If o is differentiable and de T,M + R is injective then any metric defined on IR" can be "pulled back" to obtain a metric on M. One such metric is the standard dot product: (+-)) R x R" + R. The metric inherited by .M from R" via pullback is defined as:
g TTMM T.M dq,dp, R" R" (m)), R.
In words, the inner product between tangent vectors of M can be measured by first mapping these vectors to R", and then taking their usual Euclidean dot product. If the manifold M is endowed with local coordinates (Z1,,.., Tm), the tangent space at a point p E M has a natural basis (p) = شz, :.... V.n Jam E T.MM corresponding to infinitesimal changes on the manifold as a function of infinitesimal changes in any of the local coordinates. that case, the metrie g can be written, at particular point, as a matrix G(p) E mm whose (i. entry corresponds to the inner product between (p) and (p), that is Gp)(p) - (Vv((p), vp() = (du((p)v<())
B_
(1p(p))((p)) Importantly, the metric carries all the information about the intrinsic geometry of the manifold (MM) S R For example, it captures the local curvature and warping of the geometry of the data manifold induced by the mapping y. We next show how the intrinsic geometry of the hidden-layer manifold ((M) can provide insights into the computations performed by he network.
Deep neural network representations. We start by providing intuition in the setting of deep neural networks with denendeende To this end, we built simple feedforward network trained on a binary classification task. The inputs were drawn from a circular manifold embedded in a 2D plane and the network was trained map angles 0 E (0
and 0 [T,, 2T) to -1 (Fig. 2a; S8.1). l). A simple 3-neuron network is sufficient to solve this task, and visualising the activation the neurons in response to all inputs showed that the network learned a non-trivial representation of the input manifold (Fig. 2b).
Since this is a 1-d manifold, the metric has single entry describing the magnitude of the change du dx = l| dz dx I|2 where dx dx
in neural activation given a small change in the angle: Goo de W Wiagg(á' (Wcco(()) (0)))) and dx 1- sin(O), cos(0)) for hidden layer activation de <(Wx) and an input x = [cos(()) sin( Visualising the pullback metric as gridlines on the input manifold by integrating /Go over 0 highlights that space was warped (stretched) around the decision boundaries on the circle (Fig. 2c). This means that near the decision boundary, small changes in the input angle lead to large changes in the internal representation corresponding to changes from one binary output to the other. In comparison, the untrained network did not show this effect (S8.1).
dx
The pullback metric captures fea- tures of computation a. Network with one hidden layer and tanh nonlinear- ity trained to map x = [cos(O), sin)() to y -o 1 (m) b. Activation of the three hidden units of the network in re- sponse to the inputs. Metric learned by the network - represented as gridlines on the in- put manifold - illustrating that the manifold has been warped around the class boundary.
Figure 2:
Input mantfoks
Target autoo
Input inputangle
z
AO Goo
23
Leamnd
Representaaonell
metric
Cless
boundary
21
0
D.5
22
Hence, the computations performed by deep neural network can be understood via the warping of the internal representations of its input variables. The next section generalises this to dynamical systems, where the inputs, and therefore the representation itself, are tare-varying.
3

---

3 Dynamical systems are constrained by their input topology
Dynamic neural representations. In the static network considered above, the activation of the hidden units was naturally constrained to the same manifold as the input itself. Thus, the topology of the input manifold places strong constraints on the hidden-layer topology. We next generalise this relationship to that between the manifold of time-varying inputs to dynamical system and the manifold to which its state is constrained. Throughout, we consider dynamical system of the form:
(t) f(x(t), u(t)), x(0) Xg E R"; x(1) E R",
f: x R R
Our central theorem says that if the inputs lie on a low-dimensional manifold then so does x(t).
Theorem 3.1. Consider dynamical system as above, and let u E M where M is an m-dimensional manifold. Then x(1) E N where N P(M x R) and P a projection map.
This theorem explicitly characterises the dimensionality of the manifold of states of the system. Corollary 3.2. If M is m-dimennional then N is at most m + 1-dimensional.
We stress that the dimensionality of the imput manifold need not match the dimensionality d of the space in which the inputs lic at any particular time point. For example, constant inputs of varying angles might be embedded u(t) [cos(0), sin(0) (0)] and therefore have d 2 and
1 (i.e. (m = 20) yet has d = 1 (i.e. > d). Theorem 3.1 shows that rather than d determines the dimensionality of the dynamical system's 's manifold. In particular, in many but not all tasks m is low or (Table 1).
m < d). contrast, the space all possible Morse codes is
RNN models of neuroscience
Table 1:
Tasks modelled using RNNs tend to have low m while the d can vary based on modelling choices.
Task:
Working Decision
making 39 Timing Navigation 50
memory 48
m
2
OC
2
2
Theorem 3.2 provides clear result regarding the dimensionality of the manifold of solutions dynamical system. This manifold is also a submanifold of the state-space RR.
Corollary 3.3. Let
Ju. ke(m) a basis of T.M. Define the solution of the dynamical system:
p(u, t) =xo + f(x, f(x, u(F))dT, P: XMMRR-1 R
a Then if span { at de
{di a ou. kE|m)
= gm for all p then N is an immersed submanifold of R".
Theorem 3.2 and corollary 3.3 are illustrated in figure 3a. Often, one will parametrise the input functions with real-valued variables Ihm (i.e. in local coordinates of M), with the dynamics then being: x = f(x, (f)). Then, this basis can be more concretely written as axx 8x du E R". Our numerical analyses will depend on this basis and the metric resulting from it. In supplementary materials $7 we extend this result to: i) the case where the dynamical system defined on another space than the Fuclidean space ii) the case where the initial state of the system also varies on low-dimensional manifold iii) the effect stochasticity in the dynamics. Similarly to the static case, natural metric can be defined this manifold characterised via an adjoint dynamical system:
Theorem 3.4. In local coordinates, the pullback metric on N is GN = UUT with U = [[f, 1, 0), [a, O, I]] where a, dri 1 follows the :ynamics':
0x
à = Jra + h(xx t, k)
where Jr is the Jacobian f and h a control term. case that will be of particular interest to the rest of this paper that of RNNs. this case the metric is given by the following theorem.
Corollary 3.5. Consider the RNN dynamical system: x = f(x, u(t)) W ((x) x + Bu(t) with
u M, then the metric on the RNN manifold in a particular coordinate chart is given by:
[| 2 ATA] AT
duk (1), di
GRNN=
A= (W>( T)A
A(0) nxm
ATA
From here on, ignore time dependencies for notational clarity.

---

Low-rank RNNs constrain the embedding dimensionality of the manifold. Recent work has focused on studying dynamical systems with low-rank weights. 51 In particular, it has been shown that the state of low-rank RNNs is naturally constrained to low-dimensional space. 52 Indeed, if the weights of an RNN are rank r, meaning that W = - a; tte dynamics of the RNN upon receiving dddiimensional input (t) can be written as x = -x-+ x Ei= ( E b,u,(1) which is linear combination of the a's and b's (plus a leak term). This means that the state of the system will be mostly constrained to the r + d-dimensional subspace spanned by the ax and b with rapidly decaying dynamics in all other directions. Thus, the low-rank RNN framework places constraints on the linear embedding dimensionality of the state while our framework places constraints on the intrinsic dimensionality of the nonlinear manifold of states. The two frameworks could naturally be combined order to design RNNs with specific intrinsic and embedding dimensionalities, by choosing the proper input manifold and rank of the weights.
Attractors have rank-deficient metrics. Another common geometric perspective on dynamical systems comes from studying attractor manifolds. Here we show how the metric can be used to understand the computational mechanisms behind classic attractor RNN model. This RNN consists of three units receiving two constant-in- -n-time inputs (Fig. 3b). The two inputs are correlated, such that the set of all possible inputs forms line manifold. Starting at a fixed initial state the network draws a trajectory which depends on the value taken by the input before settling on a line of fixed points (Fig. 3c). At time = O, the state the system is independent of the input and therefore the metric is rank-deficient with a single non-zero entry given by Gu f(x, (00))f(xx u(0)) where x = f(x, utt) = W o(x) - x + u(t) and zero entries 0- = Gtu = Gut = a T f 0 f and 0 = Gun a a()) 0 O. When + co, because of the way the weight matrix W is chosen, the system converges to steady-state meaning that Gt - 0. However, the state to which is converges is dependent on the value taken by the input and therefore Guu 0 (Fig. 3d).
Thus, dynamical systems lie on low-dimensional manifolds whose topology is constrained by their inputs and whose geometry reflects the computations they perform. Over the next sections we show how this provides novel insights into classic RNN models.
b
Trajectory
Task Input manifold
Tone
G
8xx Gu
Gie
G+
0.x
G.
Inyut maniladd
012
O.C
11.
GL
Guv
get Det
Food pointss
M
Inttial state
xit, tr Au)
Fixed points
81
10
Dynamiccal system manitold N
0.4
0.4
E1
Ea
2.
T&N
0.4
044
xX6 g)
Initial state
0.50
0.0
17
Eg
Tangent space
Time
Time
Time
Figure 3: Dynamical systems' states lie on manifolds whose geometry capture task computations. a. Schematic of how manifold of time-varying inputs generates dynamical system manifold. Each point u on the input manifold M is a time-varying function. The integral function maps this input function the time-varying solution to the dynamical system. The tangent space of the dynamical system manifold is thus spanned by two vectors corresponding to small changes in the input parameters x (given by the adjoint a) and small changes in the time point (given by f). b. E-I network of decision making, whose captures the transition from the initial state to a line of attractor fixed points. The metric the inner product between the tangent vectors. e. Single-neuron activity across different trajectories.
iitttant-intimme inputs lic on a line manifold. c. The metric
Contextual inputs warp neural manifolds
Biological and artificial agents must flexibly adapt to changes in task context to focus on relevant information while ignoring irrelevant inputs. Contextual evidence integration is classic neuroscience task which has been modelled extensively using R 23,54,55 Previous work has analysed this task by studying local lincarised dynamics about fixed points." 39 In particular, it has been suggested that
5

---

RNNs solve the task via line attractor manifolds. Here, we provide a fully nonlinear analysis of the dynamics and geometry of the computations underlying the solution to this task in RNNs. In particular, we show that the activity lies on separate three-dimensional manifold for each context and that, over time, the manifold warps to compress the irrelevant information.
Stochastic differential equations model noisy evidence integration. We trained an RNN on a classic contextual binary evidence integration task with dynamics:
dx = (W (()) - x + b1 + ba + C1 + C2 (t + Bdw, x(0) =0
where x(t) E R and B [b,, by] such that dW e R2 a Wiener process increment. In each
context ctx = i, the network is trained to output the sign of the relevant input u, at the final time
point = 10:
ctx E {1,2), 4 E [-0.2, 0.2] lew (W, D) = llv(t= = 10) sign( un 1ox=1 + 22
where y(t) = (tt)) and D E 1x is decoder. Thus, the network must learn to discard the irrelevant input in each context. The input manifold is generated by varying the two inputs (u1, U2) for a fixed realisation of the noise (Fig. 4a; right).
The network solves the task by suppressing output variability in response to irrelevant inputs. For given context, the input naturally lies on two-dimensional manifold defined by the coordinates Wi and U2 (Fig. 4a left). Thus, the neural activity lies on a three-dimensional manifold generated by the two input dimensions and the time direction. Furthermore, switching between contexts generates two such manifolds (Fig. 4b). As in the previous section, the tangent space of this manifold has a natural basis given by 0,x, O,, xu,,x. To understand how the network processed relevant and irreeevat information in each context, we asked how the readout of the network changed when varying the state of the network along each basis vector. We found that, as time progresses, varying the relevant, but not irrelevant, input leads to large change in the readout (Fig. 4c). We further illustrate this by plotting the output of the network under fixed realisation of the noise process while varying the irrelevant input (Fig. 4d).
The metric warps the manifold to discard irrelevant inputs. Thus, the trained network is able to flexibly discard irrelevant information in each context. We next sought to understand the geometry behind this computation. To this end, we visualised the manifold at different time points two- dimensional subspace spanned by two neurons. We found that as time progresses, the direction on the manifold corresponding the irrelevant input was compressed (Fig. 4e). To verify this insight at the network level, we computed the metric over the three intrinsic dimensions of the manifold. We found that, initially, the metric's diagonal entries were roughly equal, suggesting an equal sensitivity to changes in the two inputs and the time dimension. In contrast, near the readout time, i) the time component of the metric had converged to near zero, suggesting that the network had reached steady state, and ii) depending on the context, the component of the metric corresponding to the irrelevant input had significantly decreased, suggesting that the irrelevant input no longer influenced the state of the network (Fig. 4e). This confirmed that, as time progresses, the neural manifold is compressed along the irrelevant input direction. To visualize this, we computed geodesics under the metric at early and late time points, which confirmed that space had been warped compressing the irrelevant direction and by stretching the manifold near the decision boundary along the relevant direction (Fig. 4f). Moreover, as time progressed, two of the eigenvalues of the metric corresponding to the time and irrelevant input component - decayed to zero (Fig. 4g), suggesting that the manifold had become closer to -i-dimmesioonll by the decision time.
the manifold, both by
The basis of the tangent space provides insight into neural connectivity and responses. Fi- nally, we sought to relate these geometric results to the connectivity and dynamics of the network. Consistent with recent work suggesting that RNNs trained on low-dimensional tasks have low-rank connectivity," 23,55 we found that the changes in weights between the first and last training iteration AW = WK- - Wo (and the weights themselves, S8.3) were low-rank (Fig. 4h). This suggested that the dynamics of the system could be understood in terms of small number of dynamical modes.
Classic work based on linear(ised) models of the task has argued that dependtt alignment of dynamical modes, or "selection vectors" (left eigenvectors of the weight matrix), with relevant inputs underlies the solution to the task. 39 We asked whether a geometric analysis of the manifold could reveal a similar mechanism. Indeed, we found that the tangent vectors corresponding to each input were initially both partially aligned to the leading right singular vector of the weight matrix, but over time the alignment of the relevant input's tangent vector increased while the irrelevant input's tangent

---

between the time- and Ingout moniffld
Contesd Context
Output
Turget
Context
nolse
decoder
Melcon
Contio
Target 2
Colour 141
Time
Timee
Iniic
Attractor plane
Context
Outpur
Verisbiity irelevant inpur
Time
Akyiment
00
x,2x x.
y
0.1
Ou, x-d
-0.1
272
mmm
Time
Contoxt Conteot
trajoctory
8xx
Singlar value AW 1
cu
012
U.A
22
an
0,,
0,
a1
P.
Oup Vi
0.26
8xx
Bun
Ove*
Connod
Index
T=02
T= 10
Context
Context 1.5
Context
Same contood
Oppoeto agnime
Sgemouus
metiic
1.0
1.0
Context 2
0.5
10
Time
Time
9,
1/2
[0
PLX
Figure 4: Neural manifold warping of irrelevant inputs during contextual evidence integration. Task schematic: the task consists of two noisy inputs at different expected magnitude. Depending on the contextual cue, the network has to output the sign of the relevant input. Right: example outputs under the same noise realisation across different magnitudes of the relevant input. b. Because the input lies on a 2D manifold, neural activity lies on a 3D manifold (here shown in the space of three manifolds, one for each context. These manifolds are bounded by the relevant (but not irrelevant) input
neurons). Further, there planes of attractor fixed points (see panel e). c. Changes lead to changes in the decoder output at late (but not early) time points. Error bars across the irrelevant input. d. This ensures that the output is not affected by variability in the irrelevant input. e. 2D time-slices of the manifold. The metric goes from being everywhere diagonal with equal diagonal entries at early time points to having a near-zero time component (because the network has converged an attractor state) and a dominant input space component that depends the context. Geodesic gridlines under the pull-back of the metric at two different time points highlights that space becomes stretched near the decision boundary along the relevant input g. Eigenvalue of the metric (normalised to 1, error bars across the irrelevant input). The manifold becomes Secudo-Riemanee such that only the largest eigenvalue (corresponding to the relevant input) is large. h. Alignment of the largest eigenvector of the weight updates with each basis vector of the tangent space. Over time largest eigenvector becomes more aligned with the relevant input. i. The change activation of neurons following change along the each basis vector of the tangent space follow Gaussian distribution. The covariance of this distribution is porrelated/anticcoreehe relevant-input but not between the time- and irrelevant-ing -nnput
at large times,
vector decreased (Fig. 4h). Thus, context dependetd alignment of recurrent dynamical modes be captured through an analysis of the low-dimensional nonlinear geometry of the system, without linearisation around fixed point manifolds.
Finally, to understand how individual neurons participated in these low-dimensional dynamies, asked how their responses were distributed with respect to the tangent space of the manifold.
7

---

Individual neurons' responses were approximately Gaussian distributed along the tangent vectors, suggesting a highly distributed embedding of the www-dmmnnsioall manifold in neural state space (Fig. 4f). Furthermore, neurons participated in the relevant input and time directions in a correlated manner, while participation in the irrelevant input and time directions was uncorrelated, suggesting a highly structured embedding of the relevant input dimension of the neural manifold in state space together with a disordered embedding of the irrelevant input dimension.
Overall, our analysis suggests that dynamical neural representations can flexibly warp based on contextual cues to discard irrelevant information.
Working memory uses a hyper-torus with dynamic geometry
In the previous section we saw that contextual cues can warp the manifold to discard irrelevant inputs. Here we hypothesize that this dynamic warping mech- anism can be used to encode and retrieve sequence of inputs. Recent studies have characterised the neu- ral geometry of biological and artificial 48,58 60 networks performing "working memory" tasks in which information is stored in the activation of net- work and retrieved at a later time point. Sequentially presented stimuli have been suggested to be stored in orthogonal subspaces in neural activity. 36 In addition, neural representations of stimuli drawn from contin- uous input manifold have been shown to be warped into set of discrete attractors." 58.61 However, how neural geometries encoding multiple stimuli are dy- namically reoriented or warped during the encoding, delay and retrieval of working memories remains un- clear. At two extremes, stimuli encoded in orthogonal subspaces may be retrieved sequentially by dynam- ically realigning each subspace to a fixed decoder, or the geometry of the neural representation may be dy- namically warped to increase the projection of each subspace onto the decoder (Fig. 5).
Inputs
Cutputs
Tiee
0
dolay
02
Hypothesis 1: Warping
Aazzda Suiiginc
Hypothesis 2:
Rotation
Cro stanie cree
Time oulput
Time oulput
Task where a network has to re- member two sequential inputs and then out- put them after a variable delay. Possible task solutions: i) dynamically warping the torus to compress the irrelevant input's 's representa- tion or ii) realigning the torus' relevant input encoding direction to the decoder subspace while preserving a fixed intrinsic geometry.
Figure 5:
RNNs solve the task using curved geometry. We tested these hypotheses in RNNs trained to perform a sequential working memory task. The RNN se- quentially received two inputs drawn from circular manifold, and was required output the same se- quence after a variable delay (Fig. 5 top):
* = -Wo(x)
+ Bu(t) + (t)c, L(W, I (ID((x(t
h) - =u(t)]
with u(t) |cos (t). sin (t)) where 0(t) was piece-wise constant, and E 3.55 was the delay period, ended by pulse go,, (4). Applying PCA to the activity at different points revealed a toroidal manifold with dynamic geometry (Fig. 6a-b). Yet this low-dimensional visualisation is insufficient to fully characterise the RNN's representational geometry, which requires analysis of the metric.
The metric dy dynamically warps the torus to represent time-specific relevant information. To test for warping of the neural manifold we computed the Gaussian curvature of the torus at different time points (Fig. 6c; S8.4). We found that the torus had a non-ilat geometry, including both positive and negative curvature which was highly non-uniform over the torus (Fig. 6c-d). Furthermore, the metric revealed that: i) during the input period the torus' shape was formed ii) over the delay remained stable and iii) during the retrieval phase the RNN selectively compressed the stimuli not immediately retrieved (Fig. 6e). Finally, we looked at how these changes intrinsic geometry related to the extrinsic embedding of the torus in the state space. First, the decoder was selectively aligned to the basis vector encoding the relevant stimulus at each time point during retrieval (Fig. 6f) and the subspace spanned by the top two principal components was stable across the delay (Fig, 6g).
8

---

Storing more memories requires higher-dimensional manifolds. Finally, we ask how these results generalise to storing more memories. First, we proved (S8.4) that the activity of the RNN during the delay period lies on a hyper-torus whose dimension equals the number of items stored, and that, furthermore, to optimally encode the stimuli this torus must be embedded in the state space (as opposed to immersed). Computing the geodesic distance on the hyper-torus w.r.t. reference combination of angles revealed how the hyper-torus was compressed along irrelevant directions at different time points to retrieve different memories (Fig. 6h-i).
In sum, we mathematically show that sequential working memory relies on higher-dimensione representations. In RNNs, a combination of dynamical changes in the intrinsic and embedding geometry of the manifold occurs during retrieval of these memories.
hppu
1rpu2
Delay
Output
Cutput 2
Outizut
Qupput
2
Gos,, (t(G)
1.0
b
Impul
Delay
0.5
PCs
0.0
Time
Gauesiar curvelure
V >
Output
Subspace sim.
V2
3
0xx
Output
Go
X
Curv. <0
8,.
0.3
akgrmmnt
Decoder
Time
Delay
0xp
PCs
Oct
Do
Oe:
0.0
Time
Time Go
Output
Output
Output
Outout 2
Ga (tr(G)
Durput
Outout
0200 0001
1099-03
82
ane
out Stde
Arc: length
Time
Time
Figure 6: Recalling memories requires dynamically warping their neural representations.
a. Example of network input and output. b. Different time slices of the RNN manifold, shown projected on the first three principal components (PCs) (inset two PCs). After the first input comes on the state any particular time point is lies on a 1D manifold encoding the input; after the second input comes the state is constrained to torus encoding both inputs. Gaussian curvature of the torus at output onset (t = 6). The torus has positive and negative curvatures at different points. Right: Two different pieces of the torus projected in the PC space displaying positive and negative curvatures at the start of the delay period (f 2). d. Metric of the RNN manifold at the start of the delay period 2). Average stretching of the torus, quantified by the ratio of one of the diagonal entries of the metric to its trace. Top: Metric averaged over the torus. The torus goes from being warped along the encoding of the first then second angle during retrieval. f. Alignment of the basis of the tangent space to the decoder || DDiaa (ø' (x(x))0o, x Blue is 0 and yellow is 02. Subspace similarity between the spaces spanned by the first two PCs computed any pair of time points. h. Ratio of individual entries of the diagonal of the metric to its trace. i. Arc length from a reference angle. The hyper-torus stretched dynamically through the recall period compress the irrelevant angles.

---

Discussion
Conclusion. In this work, we developed mathematical framework extending previous applications of Riemannian geometry in machine learning to dynamical systems receiving time-varying inputs on low-dimensional manifolds. By deriving the pullback Riemannian metric on the manifold of system states, we showed that the geometry of a dynamical system reflects its computations, linking representational geometry to computation-througe through-dynamics. This framework thus provides mathematical description of how dynamical systems transform input manifolds into neural activity manifolds whose topology and geometry are constrained by the structure their inputs.
In Section 3, we used this framework to show that contextual inputs warp the geometry of neural manifolds in RNNs trained on decision-making tasks. Specifically, we found that the metric com- presses directions corresponding to irrelevant inputs while preserving those corresponding to relevant task variables, enabling flexible context-depene -depended computations. analysis to sequential working-ment memory tasks and showed that RNNs encode and retrieve multiple memories through dynamic warping of sigher-dimensionlll manifold, specifically hyper-torus whose intrinsic and embedding geometry evolve over time. Together, these results demonstrate that warping a ubiquitous feature the computations performed by RNNs across distinct task domains.
Section 4, we extended this
Limitations. The relationship between geometry, dynamics and connectivity is degenerate: the same geometry (i.e. the same metric) can arise through different network architectures. Future work could investigate whether networks with identical geometries but distinct connectivity and dynamics can implement distinct classes of computations, instead constitute a degenerate class of solutions the same problem. A second limitation is our analysis of the influence of noise on geometry. In particular, we treated noise in pathwise manner; we studied the manifold generated under fixed realisation of the noise process. The distribution of manifolds generated by random noise realisations remains to be characterised, and could provide insight into the geometry of computations on stochastic data.
Broader impacts. Our framework is applicable to wide range of dynamical systems. In particular, methods infer dynamical systems from biological physical data using machine learning tools have been widely adopted, and the Riemannian geometry of these systems could analysed to interpret the behaviour of these data-driven dynamical systems. particular, the computational mechanisms identified here in task-trained RNNs could be investigated in RNNs fit directly to neural data in order test hypotheses regarding the geometry of computation large- scale neural recordings.
10

---

References
Cunningham, J. P. and Yu, M. 2014, Dimensionality reduction for large-scale neural recordings. Nature neuroscience 1, 1500-1509.
Chung. S. and Abbott, 2021, Neural population geometry: An approach for understanding biological and artificial neural networks, Current Opinion in Neurobiology 70, 137-144.
Cohen, U. et al. 2020, Separability and geometry object manifolds in deep neural networks, Nat Commun 6(11).
DiCarlo, J. J. and Cox, D. D. 2007, Untangling invariant object recognition, Trends in Cognitive Sciences 11(8), 333-341.
5. Li, K. et al.. Emergent World Representations: Exploring a Sequence Model Trained on Synthetic Task, 2024.
Fefferman, C., Mitter, S., and Narayanan, H. 2016, Testing the manifold hypothesis, Journal of the American Mathematical Society 29(4), 983-1049.
7. Hauser, M. and Ray, A. 2017, Principles of Riemannian Geometry in Neural Networks, Advances in Neural Information Processing Systems, ed. by Guyon, I. et al., vol. 30.
8. Chen, R. T. et al. 2018, Neural ordinary differential equations, Advances in neural information processing systems 31.
9. Kidger, P. 2022, On neural differential equations, arXiv preprint ar arXiv:2202.02335.
10. Lee, K., Trask, N., and Stinis. P. 2022, Structure- preserving sparse identification of nonlinear dynamics for data-driven modeling, Mathematical and Scientific Machine Learning, pp. 65-80.
Strseture-preserving
11. Yu, Y. et al. 2024, Second-order forward-mode optimization of recurrent neural networks for neuroscience, Advances in Neural Information Processing Systems 37, 92101-9102 92119..
12. Yang, G. R. al. 2019, Task representations in neural networks trained to perform many cognitive tasks, Nature neuroscience 22(2). 297-306.
13. Duncker, I.. ct al. 2020, Organizing recurrent network dynamics by task-computation to enable continual lcarning, Advances in neural information processing systems 33, 14387-14397.
14. Pellegrino, A., Cayco Gajie. N. A., and Chadwick, A. 2023, Low tensor rank learning of neural dynamics. Advances in Neural Information Processing Systems 36, 11674-11702.
15. Donå, J. et al. 2022, Constrained physical-statistics models for dynamical system identification and prediction, ICLR 2022-The Tenth International Conference on Learning Representations.
16. Weichwald, S. al. 2022, Learning by doing: Controlling dynamical system using causality, control, and reinforcement learning, NeurlPS 2021 Competitions and Demonstrations Track, pp. 246-258,
17. Atanackovic,
2023, Dyngfn: Towards Bayesian inference gene regulatory networks with gflownets. Advances in Neural Information Processing Systems 36, 74410-74428.
18. Badia-i-Mompel, P. et al. 2023, Gene regulatory network inference the era single-cell multi-omics,
Nature Reviews Genetics 24(11), 739-754.
19. Saharia, C, et al. 2022, Photorealistic Text-to- Text-to-Image Diffusion Models with Deep Language Understanding,
Advances in Neural Information Processing Systems, ed. by Kayejo, S. et al., vol. 35, pp. 36479-36494.
20. Sussillo, D. 2014, Neural circuits as computational dynamical systems, Current Opinion in Neurobiology
25, 163.
21. Sussillo, D. and Barak, O. 2013, Opening the Black Box: Low-Dimension Dynamics in high-Dimenssioal Recurrent Neural Networks, Neural Computation 25, 626-6499
22. Smith, J. Linderman, S., and Sussillo, D. 2021, Reverse engineering recurrent neural networks with Jacohian switching linear dynamical systems, Advances in Neural Information Processing Systems 34, 16700-16713 16713.
23. Valente, A., Pillow, J. W., and Ostojic, S. 2022, Extracting computational mechanisms from neural data using low-rank RNNs, Advances in Neural Information Processing Systems 35, 24072 24086.
24. Khona, M. and Fiete, I. R. 2021, Attractor and integrator networks in the brain, Nature Reviews Neuroscience
23, 744-766.
25. Hénaff, O. et al. 2021, Primary visual cortex straightens natural video trajectories, Nature Communications 12(5982).
26. Saxena, S. al. 2022, Motor cortex activity across movement speeds is predicted by network-level strategies for generating muscle activity, eLife 11, e67620.
27. Machens, C. K., Romo, R., and Brody, C. D. 2005, Flexible Control of Mutual Inhibition: Neural Model
of Two-Interval Discrimination. Science 307(5712), 1121-1124.
28. Wong-Lin, K. and Wang, X.-J 2006, A Recurrent Network Mechanism of Time Integration in Perceptual Decisions, The Journal Neuroscience 26, 1314-1328.
29. Stokes, M. G. et al. 2013, Dynamic Coding for Cognitive Control in Prefrontal Cortex, Neuron 78(2). 364-375.
11

---

30. Darshan, R. and Rivkind, A. 2022, Learning to represent continuous variables in heterogeneous neural networks, Cell Reports 1).
31. Ságodi, A. et al. 2024, Back to the Continuous Attractor, The eiihtt- Annual Conference on Neural Information Processing Systems.
32. Stanczuk, J. et al. 2022, Your diffusion model secretly knows the dimension the data manifold, arXiv preprint ar2iv::212..166
33. Nam, A. et al. 2023, Discrete, compositional, and symbolic representations through attractor dynamics. arXiv preprint arXiv:2310.01807.
34. Rohrhofer. M. et al.. On the Role of Fixed Points Dynamical Systems Training Physics-Informeet Neural Networks, Transactions on Machine Learning Research.
35. Aitken, K. et al. 2020. The geometry of integration in text classification RNNs, arXiv preprint arXiv:2010.15: 15114
36. Krishna, N. H. et al. 2023, Sufficient conditions for offline reactivation in recurrent neural networks, The Twelfth International Conference on Learning Representations.
37. Schaeffer, R. 2020, Reverse-engincering recurrent neural network solutions to a hierarchical inference task for mice, Advances in Neural Information Processing Systems, ed. by Larochelle, H. et al., vol. 33,
pp. 4584-4596
38. Cenedese, M. et al. 2022, Data-driven modeling and prediction of non-lincarizable dynamics via spectral submanifolds, Nature communications 13(1), 872.
39. Mante, V. et al. 2013, Context-dependent computation by recurrent dynamics in prefrontal cortex, nature 503(7474), 78-84.
40. Aoi, M. and Pillow, J. W. 2018, Model-based targeted dimensionality reduction for neuronal population data, Advances in Neural Information Processing Systems, ed. by Bengio, S. ct al., vol. 31.
41. Benfenati, A. and Marta, A. 2023. A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations. Neural Networks 158, 331-343.
42. Katsman, 1. et al. 2023, Riemannian Residual Neural Networks, Thirty Thirty-seventh Conference on Neural Information Processing Systems.
43. Brandon, J., Chadwick, A., and Pellegrino, A. 2025, Emergent Riemannian geometry over learning discrete computations on continuous manifolds, arXiv preprint arXiv:2512.0019)
44. Aubry, M. and Russell, B. C. 2015, Understanding deep features with computer- generated imagery, Proceedings of the IEEE international conference on computer vision, pp. 2875-2883.
45. Tai, K. S., Bailis, P., and Valiant. G. 2019, Equivariant transformer networks, International Conference on Machine Learning, pp. 6086-6095.
46. Tennenholtz, G. and Mannor, S. 2022, Uncertainty estimation using riemannian model dynamics for offline reinforcement learning, Advances in Neural Information Processing Systems 35, 19008-19022
47. Silva, M. F. da, Dangel, F., and Oore, S. 2025, Hide & Seek: Transformer Symmetries Obscure Sharpness & Riemmanian Geometry Finds It, arXiv.
48. Cueva, C. J. et al. 2021, Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes, arXiv preprint 1 1.01255.
49. Wang. J. ct al. 2018, Flexible timing by temporal scaling of cortical responses, Nature neuroscience 21(1), 102-110.
50. Redman, W. et al. 2024, Not so griddy: Internal representations of RNNs path integrating more than one agent, Advances in Neural Information Processing Systems 37, 22657-22689
51. Thibeault, V., Allard, A., and Desrosiers, P. 2024, The low-rank hypothesis of complex systems, Nature Physics 20(2), 294-302.
52. Mastrogiuseppe, F. and Ostojic. S. 2018, Linking connectivity, dynamics, and computations in low-rank recurrent neural networks. Neuron 99(3), 609-622 -23.
53. Wong, K.FF. and Wang, X.-. 2006, A recurrent network mechanism of time integration in perceptual decisions, Journal of Neuroscience 26(4), 1328-1328.
54. McMahan, B., Kleinman, M., and Kao, J. 2021, Learning rule influences recurrent network representations but not attractor structure in decision becision-making tasks, Advances in Neural Information Processing Systems 34, 21972-1222 21983.
55. Dubreuil, A. et al. 2022, The role of population structure in computations through neural dynamics, Nature neuroscience 25(6), 783-794
56. Xie, Y. et al. 2022, Geometry of sequence working memory in macaque prefrontal cortex, Science 375(6581), I), 632-639.
57. Panichello, M. F. et al. 2024, Intermittent rate coding and cue-specific ensembles support working memory. Nature, 1-8.
58. Panichello, M. F. and Buschman, T. J. 2021, Shared mechanisms underlie the control of working memory and attention. Nature 592(7855), 601-605
12

---

59. Liebe, S. et al. 2025, Phase of firing does not reflect temporal order in sequence memory of humans and recurrent neural networks, Nature Neuroscience, 1-10.
60. Ritter, I., and Chadwick, A. 2025, Efficient Working Memory Maintenance via High-Dimensionn Rotational Dynamics, hioRxiv, 2025-09.
61. Panichello, M. F. et al. 2019, Error-oorrecting dynamics in visual working memory, Nature communications 10(1), 3366.
13

---

Supplementary Materials
S7 The representational metric of dynamical systems
In this section we introduce the mathematical framework used throughout the main manuscript. In order for this work to be self-contained, we first briefly review the relevant concepts of differential and Riemannian geometry. To provide intuition in a mathematically simplified setting, we study deep series of geometric transformations, showing that this view can provide insights into the computations performed by the network. We then provide our main contribution: a full derivation of the geometry of dynamical systems' manifolds. Finally, show concrete application of these mathematical results for continuous-time RNNs.
neural networks
S7.1 Differential geometry of deep neural networks
The results of this subsection mostly follow from elementary differential geometry (e.g. we refer the reader to!) ), and we therefore state them without derivation.
The topological and differentiable structure of deep neural networks. In this section we consider a generic depth-I neural network of the form:
5 ft JI-1 o.... fi R R
where f& is the map from hidden layer k - 1 to hidden layer k. For example, fz lanh(W,2; + b) = 2 where 2₁ E IK* is the vector of activations of the neurons in the kth hidden layer. Following previous work, 2 we consider the input this network as living on liw-dimensional manifold:
(x((((p)) = (x) Zs: PEM
where to is embedding of M in RT The following proposition gives clear characterisation of the topology of the manifold over which z lies. In particular, if yps : R - IR" is i) continuous ii) invertible and iii) its inverse is continuous then it is homeomorphism.
Proposition S7.1. Let E Re be constrained to (M), embedding of M in R=e and let sp be as defined above. If sp is a homeonorphism then zą sP (x) is also constrained to M.
Importantly, & is a simple real function, and its invertibility and continuity can be verified via basic calculus. Thus, x and z are on the same manifold, simply represented differently in Io and R".
In general, the invertibility and continuity of the network transformation are mostly independent of the parameters of the network. For example, if the network is of the form fx o(W,x + b) where is an element- dementt-wise nonlinearity. then if à is (element-wise) homeomorphic (e.g. tanh, softplus, leaky-ReLU, but not ReLU),, and W₂ is full rank, and the network doesn't 't have a bottleneck layer, then these conditions are satisfied. In particular, W; is full rank with probability if it is initialised with entry-wise i.i.d. Gaussian or uniform weights. Thus, provided the rank of Wa does not change over training, the topology of a trained network will match that at initialisation, and hence does not reflect the computations learned by the network. However, in some instances the topology may indeed change: recent work has highlighted the ubiquity of low-rank networks, and depending on the objective, the learned weights may be of a rank lower than the minimum embedding dimensionality of the manifold trivially one could think of network trained to map all its inputs to O. Nevertheless, in general it is the geometry of the neural representation that provides most insight into the computations learned by the network, as this geometry depends continuously on the parameters of the network whereas the topology is invariant to large classes of parameter changes.
There are multiple routes to characterising the geometry of a manifold. Here we will specifically consider smooth manifolds. In particular, such smooth manifolds have a tangent space TrM at any point p .M.
Proposition S7.2. Let Pp : R + R differentiable with Jacobian Ja E R If Jn is full matrix rank and an embedding, then d(yps 1) Jzdv(v): T,M - Tp)((()) surjective and M is an immersed submanifold of m
We note here that the differentiability and homeomorphicity conditions are overlapping but not restrictions of one another. In particular, even when yok is not bijective dipx can still be surjective
14

---

and therefore its Jacobian full rank. Similarly, Ph bijective and differentiable does not imply dipk surjective.
Riemannian geometry provides insights into network computations. So far the structure we'r ve explored had geometry, in that did not define notion of distance between points the manifold. For this, we must add a metric to the manifold.
Definition S7.1. A metric g is function defining the inner product between vectors of the tangent space of manifold. The metric is Riemannian if it satisfies: g TpM x TpM + I such that g(v, v) > v + T and g(v1, V2) = g(v2, V1)
Importantly such metric allows defining the geodesic distance between points: Definition S7.2. Let [0, T] - M smooth curve on M where 7 E RR. In particular, this implies that (t) M, and we can define the length y as:
L(7) L/ (((())
Furthermore, the geodesic distance between p1 E M and p2 E .M is defined as:
P2) = L(7), constraining 7(0) = Pl, (T)) =
This definition central our study of dynamical systems in the main text.
In figure 2 the manifold considered was the circle M = S and representing different points on the circle via their angle (p 0), the embedding of the circle in the 2-d input space was x = (0) = [cos(0). sin( However, we could have chosen different embedding, say x = [11+0.1 sin( (0), sin(f)], which in general will affect the geometry of the hidden representation Zk. Indeed, suppose the target output independent of the input geometry - as is the case in figure 2 where it is only the point the circle that determines the output then the network must perform different computations on these different input representations to perform the same output.
To characterise these transformations, we can pull back the usual metric defined on IR* which is the standard dot product, to the manifold M.
Definition S7.3. The pullback metric is defined as:
g(v1, v2) (d((px vb) (vi)) (d(p> (v)))) V1, V2 Tp
in particular d(pp o (v:) ) E IR and is the standard Euclidean dot product.
Thus, we can define a metric on M by taking tangent vectors in T,M, pushing them forward through the network, where at the kth layer they become vectors of T = R" such that we can then simply take their dot product. The value of this dot product our definition of the inner product between the original tangent vectors in M. The pullback metric represents how the tangent spaces of the manifold are transformed through the layers of the network and therefore the computations the network. In particular we can more explicitly write the pullback metric as:
g(vi, v2) (Judk(vvi) (Jzd((V22)
with
Jg = W Widiag(0 (z)-z)))) W diag(ø' ((x)((o(())dd(vvv))
where Ja E X is the Jacobian of Øs. This highlights the earlier claim that the geometry, as defined via the pullback metric, is directly dependent on the parameters Wi,.., Wi the network.
Geodesic gridlines and computations. In our networks the input metric was uniform, but in general data of the world maybe be unevenly distributed the manifold, which will affect the learned parameters, and therefore the learned representation. the case of our simple network the loss function more formally defined as:
1 (())()(0)) - Ytorgeel /
1
where h T.M x TyM is the metric on the input manifold. That is, some inputs are more heavily weighted in the loss if space is more spread around them. For example, in probabilistic setting where
15

---

instead of integrating over all possible imputs only finite sample of them is drawn at each iterations, (h) can be seen as probability distribution (up to normalisation) on the input manifold - so that some input-output pairs are drawn more frequently than others. In our case all angles were equal, so the input metric was uniform:
1 1=c (1))(((0)) - -ttrreet |
for some C E R+. But more generally non-uniform metrics on the manifold of input will lead to different learned representations since the loss is different. For example, a class being more frequent in binary classification task may change the representation: if in figure 2 we had sampled more frequently angles [0, π) than (sr, 2T). would have led an asymmetric stretching of space on each side of the class boundaries.
Thus, the pullback metric 9 gives insights into the learned representation, but the learned representa- tion is dependent on the metric of the input manifold h. Using the definition of geodesic distance, both 9 and choose to measure the distance between? In 2, 4 and 6 we showed that an insightful reflection of the computations done by the network was provided by the computing the geodesic distance using between points that were evenly apart under the h metric. That is define points P1 P2 such that d (P1,P2) = 1 and study d9 (p1, P2) where d' and d are the geodesic distance functions under the h and metric respectively.
be used measure the distance between points on the manifold. But what points to
Coordinates the input manifold provide natural input metric. So far treatment of the topology, smooth structure and geometry of deep neural networks has been relatively free of choice of coordinate system for M.. An mn-dimensional manifold is defined Rm glued together via the so-called transition functions. However, different choices of pieces of IRm can give rise to the same manifold. A trivial example is that of a line manifold: the open interval (0, 3) and (10, 16) are the same manifold, but endowed with different coordinates; there is an explicit diffeomorphism between them given by F(p) 10 + 2p.
collection of pieces of
In many cases, there is a meaningful coordinate system - at least around a point - on the input manifold. A manifold of images may for example be characterised by the position of object in the image or its orientation." 4 is insightful to understand how a neural network represents these different features. When a specific coordinate system is given, natural matrix representation of the metric can be defined.
Definition S7.4. A given coordinate system? x(p) = (2,., Im) where x M - R induces natural basis of the tangent space such that T,- can be written linear combination of them:3
m
U Ô
Zx
In this coordinate system inner products are given by:
m
g(vl, v2)
Σ
G
This higlights that the metric in a particular coordinate system can also be written as a matrix a are its entries. Furthermore, the manifold embedded in the G E Rmxm and Gi, = g( Dzy ozy)
Euclidean space:
Op
Op
= JERRR J=
E m
Opi
.... 0pm
Back to our deep neural network, the pullback metric changes from layer to layer as:
G = dvv J J+d du dd((v) JF.
JT. J..... mmm
"Th reader may be more familiar with the notation (U, up) for a chart, here we' 'This notation defines dey (ʃ 7)1- for the smooth curve 7(() defined such that xT(7) (pi( (p)p)) t,..., (p)p) and smooth function f M + H.
we've ve simply written x =
16

---

where Jy. E Ru is the Jacobian of fa. This means that:
G = d(p (v) d(p ()(() E R
Integrating along a curve on the manifold under the pullback metric in coordinates thus gives:
I(7) S ((0()) ' d(ø G ()(()))
Where now d(p (()) is usual derivative of a real function, and G3 real matrix, which can both obtained using standard calculus tools (e.g. auto-differentiaioon)
S7.2 Dynamical systems' topology
We now generalise the theory of the previous subsection to dynamical systems, where the inputs are time-varying functions. As shall see, manifold of functions naturally gives rise manifold of functions that are solutions to the dynamical system. Much of the work will lie in viewing this manifold of solutions as a submanifold of the state space.
The topology of the input manifold constrains the topology the solution to a dynamical system. Here consider more general setting than in the main manuscript where the dynamical system is defined on manifold. That is:
x()) f(x(1), u()),,
x(t) E R", u(() E ud
Furthermore, consider manifold M of input functions. That u E M and u(t) U where U is the ambient space in which the input is embedded at any particular time point. To define such manifold of functions (or more precisely of curves) we rely on the following construction:
Definition $7.5. A manifold of real curves is manifold M such that (u, E MM for u(l) u.
and u E M is u = π xu,, 1) for æ the projection map.
On particular chart, the function can be parametrised:
uk (t), x R
For example, in section 4 of the main manuscript we' ve considered uw [K1, K2, Lex=11 2] where K1, Ai were the levels motion and colour evidence. This input, fixing one of the two discrete contexts, indeed lies on 2-dimensional manifold with planar topology k E R², We formulate stronger version the theorem of the main text:
Theorem $7.3. Consider the dynamical system:
(() f(x(1), x(1) R", u(() E E M
M
where f least once differentiable arguments, then x E Am+1 (the curve) and N has the topology of mm x TR.
where the superscripts indicate the dimensionality the manifold. Theorem 3.2 of the main text therefore just special case of this theorem the dimensionality of the dynamical system is directly defined by the manifold topology. To prove this theorem we need the following lemma:
Lemma S7.4. 1 Let ý = g(y), y0) yo with stt, yo) yo + fo (Ty(T))dd the solution of the dynamical system at time starting at yo. then if g smooth, yp a diffeomorphism.
We can now prove the main theorem:
Proof of theorem $7.3. We prove this in particular coordinate chart of M and R. Then, the dynamical system can be reformulated as (by abuse of notation):
*(t) f(x(t), (t)), x(t) R.. K Rm
We can augment this dynamical system by introducing the variable y E RR" x R x R. defined as:
xtt,k) y(t) (x(t), t.
then:
y'
1, 1.
[f(y Uyatantmti (3n++1 1, y(0) [xo, 0, Ac]
17

---

which is an autonomous dynamical system. By the theorem on solution of ODE, if the r.h.s. is least once differentiable in its argument (which it is since is). there exists a unique solution to this dynamical system. Furthermore, by lemma S7.4, sp is a diffeomorphism and since k. is on an m-dimeensioal space and f in a 1-dimensional, y(t) in an m + 1-dimensional space. More generally is an m + 1-dimensional space of curves of topology Ih x R. Finally, x and are simply related by a projection x(t) = P(y(t)) = Yim-
For the sake of completeness we also show that this holds not only over a single chart but over the whole manifold. Consider smooth transition function Fig : R + Rm from coordinates of chart to coordinates of chart of M, and (o, k²) the solution the dynamical system chart i. Then ge (1, k²) = sp JR., k²) where JF.,, is the Jacobian of Fa.. Since yp is a composition of diffeomorphisms it is diffcomorphism.
Different conditions on the projection will give different properties to x, for example if P is homeomorphic then its preserves the topology of the manifold, if aP is surjective then x is on M x I an immersed submanifold of R".
We stop to address two natural questions regarding edge-cases of this result: 1) Asymptotic Regimes. Our result assumes that & IR. (transient dynamics) rather than the asymptotic limit (f -> oo). While the input-driven manifold usually matches the input dimensionality during transients, the geometry of the asymptotic invariant set can be fundamentally different. For instance, the mapping from inputs to steady-states may not be diffcomorphism due to bifurcations, and in chaotic regimes, the system may settle onto strange attractors with fractal dimensionality." 5 2) Chaos. A chaotic system affects the geometry of the manifold, not its topology. To see this, consider pulse inputs of varying magnitude to + At. the state lies on a 1-dimensional manifold parametrised
u(t) = kibr, (t). At time by K. after what it evolves autonomously. If the system has positive Lyapunov exponent, small perturbations in & result in exponentially diverging trajectories x()). Consequently, the tangent vector 0.x will grow rapidly, and the associated metric tensor entries Gr,t and G., will diverge. However, despite this warping, the topological result holds.
So far - and in the main text - we have mainly considered fixed initial conditions of the dynamical system but the previous proof naturally gives the following corollary.
Corollary S7.5. If xa E Q" and u E M then y(t) E Nm++++1 M
x
x
Q with
x(t) = Pty(t)) so that varying the initial state only adds extra dimensionalities.
Proof. The proof follows mutatis mutandis from that of theorem S7.3 additionally noticing that if Xg E Q" then yo E Q" x {0) x R..
The effect of noise can also be studied from theorem S7.3. In general, noise processes driving differential equations are sampled from an difinie-dimesiionlll manifold of functions (e.g. L Thus, the manifold of states taken by the system under all realisations of the noise may in general be the entirety of the state space. However, the following corollary states that fixing the noise realisation generates a manifold:
Corollary S7.6. Consider the stochastic differential equation dx = f(x, u(())dt + g(x)dW, Xo= x()) where g(x) E 1aaxq q is at least once differentiable in x and dW R2 are the increments of a Wiener process. Then if u E M then x(t) = P(y(t)) where P is projection matrix and y() E M x M
Proof. The proof follows a similar construction as the deterministic case. As S7.3 let y() =
(x(t), E, ki., now dy f(x, (t) + g(x, u₂ (t))dW where 1, OJ. g = Ig" T 00 Thus y follows an SDE driven by the same noise realisation. Then, using the smoothness of the solution of an SDE with respect to its initial state, 6 the rest of the proof follow the same steps as that of S7.3. O
This shows that under a fixed noise realisation the state of the system lies on a manifold. In general, for different noise realisations P may yield a submanifold an immersion or an embedding of A x R,. Future work could study the distribution of such objects under the noise, for example characterising the measure of the solutions for which it is a proper embedding or immersion.
18

---

$7.3 Geometry as input-manifold transformations
Having characterised the geometry and differentiable structure of the manifold of dynamical systems. we next turn to their Riemannian geometry. The main insight will be that the metric on the manifold can be defined in terms of the solution to an adjoint dynamical system. This will provide both theoretical insights and practical way to obtain the matrix representation of metric.
Proof of theorem 3.4. The Jacobian of y given by (ignoring time dependencies):
3x
0x
f
Jy
D2
1
0
OK
1
0T
where the submatrices are concatenated along their appropriate dimensions. The term is simply the r.h.s. of the dynamical system, so it remains to compute the term in blue. Fach column of this term is of the form dx which describes how the state of the system at a particular time point changes for an 0н. infinitesimal change in one of the input variables on a particular chart. These are given by:
0x 0
xo + L f(x(t), (,,(t,,T,ux(F))) (F))dT
0ki Ōk:
= So" dR.
f(x(t), T, u,
Now:
0x S'" of du of 0x
dr dx 0x;
0k, du 0x,
where we have omitted the arguments of each function for notational clarity. Taking the time derivative both sides and noticing that 8x
0 this defines new dynamical system:
d 0x df Ox Of Ou
0x
0
dt 0kg dx dk:
du dk:
0ky (=0
note that this form of adjoint dynamics. Indeed if define ai(() of du then we obtain the final equation:
8x Dcc
(1) and h(x,,,))
ou de,
à Jra + h(x, t,
This is controlled dynamical system, where the control term h depends the solution of the original dynamical system xt) and the input ut)).
S7.4 Some practical considerations
In this section we briefly discuss how this framework be concretely implemented in auto- differentiation framework such as Jax or Pytorch. The numerical solution of a dynamical system obtained with, say, an Euler solver, differentiable with respect the input to the dynamical system at any time point.' 7 In our work, we compute these derivatives in different ways that differ in their computational cost and accuracy:
Adjoint dynamics
Auto-differentiation
Finite difference
Time complexity Memory complexity
(nnt) ((n)
(nn O ml
O
O
Easy implementation
Accurate
In figure 3 we studied the adjoint analytically, figures 4 and 6d-h we used auto-differentiation and in figures 6e and $9a,b finite difference.
19

---

S8
Models
In this section provide the detailed description of the models used throughout the main manuscript. Jupyter notebooks implementing all analyses performed here are also provided.
S8.1 Two-layer deep neural network
In section 2 we considered one-hidden-layer deep neural network given by:
tanh(Dz) D2) E TR?, tanh(Wx b), E R W E IR D R1x3
The inputs x are constrained a eircle plus some noise during training:
x(0) [cos(0), sin( + E, E [0, 2x), ~ (N0(,1)
where N is the Gaussian distribution. The loss was given by:
J.¹ ()) 11|_d0 {_ llv(0) (-
We evaluated these integrals by discretising [0, 2x)) into 200 even bins. The parameters were initialised ways b, ((21/2 1/2). di,j U(1/3, 1/3) where 2/ is the uniform distribution and optimised using Adam with learning 0.01 over 1000 iterations.
a
C 2m Geodesics
Pre-training
-0.5
Representationa. metric
0.0
Figure S7: The representational metric changes over training in deep neural networks. Top: ran- domly initialised network. Bottom: network after training. tion of the hidden layer in response all inputs. b. Learned representa- tional metric visualised gridlines the input manifold. Arclength under either the uniform input met- ric (dashed line) learned metric (blue line) integrating from O to 0 2n. d. Same analysis across different activation functions. Al- though the intrinsic geometry varies pre-training, the post-training ge- ometry is consistent, hinting shared computational mechanism.
2-0.6
-0.8
22
21
Input metric
Post-training
L(x(0)) L(z(0))
Activa-
Repressntaat
memic
8.
21
22
in
d
Softplus
Tanh
ReLU
GeLU
training
Post- training
Decision Boundary
S8.2 E-I network
In section we considered a three- neuron continuous time RNN following Dales' law, also called an Sestatorr-Inhibiion (E-I) network. The dynamics were given by:
three-neuron
= W (xx) x Bu
The inputs were constant time and lied on 1-d line manifold and the weights were defined according to:"
a
c
W
b
== [u,
d
d
20

---

with b, c, d E R_ and u E [0. We discretized u. E [0, 1] into 200 points and evaluated the corresponding dynamical for each point. To evaluate the 200 dynamical systems we used the Heun second order numerical method until they reached steady-state, saved at 100 time points. This gave a 200 100 x 3 mesh that is visualised in panel c of figure 3.
S8.3 Contextual decision-making under noisy inputs
In section 4 we used a classic model of contextual evidence integration." 11 The model is described in the main text. The weights were initialised as waj NN g/ Vn) the other parameters were initialised as bi.j Cr = (0, 1/v2) and d ~V /n). We used a differentiable SDE solver with the Heun method and an adaptive step solver. To optimised the system we used Adam in Optax. 12 The
were:
12
9
inputt
neuuron
Ir
Iterations
Optimiser
SDE solver
PID rtol, atol 10-2, 10-3
100
0.5
0.5
0.05
5x10-5
3000
Adam
Heun
We next briefly describe the novel analyses performed here.
The basis of the tangent space. The basis of the tangent space was constructed using auto- differentiation through the ODE solver. This can be simply achieved in Jax by defining fully differentiable function o 1 x M - IR" taking points on the manifold of evidences M and a particular time point in R to return the state of the system that time point. This function can then be differentiated w.r.t. each of its arguments to obtain the basis of the tangent space at the state of the system that time point. This function is composition of i) an embedding of the manifold M in I²2 which here simply consists of u(t) = [U1, uz] for U1, E 0.2, 0.2] ii) the evaluation of the ODE under these inputs.
Variability over noise. In figure 4a (right) and we show the variability of the output of the network to variations to its input. For this we fix a particular Wiener process realisation W and evaluate the dynamical system by either fixing U2 = 0 and varying 41 E [-0.22 0.2] (for panel a) or fixing w1 = 0.2 and varying U2 E [-0.2, 0.2] (for panel d). The rest of our analyses were performed under a fixed realisation of the noise: the one where ctW(t) = 0 for all f. Although we note that future work could explore the distribution of the analyses presented here under the measure of this noise process.
Manifold plotting. In figure 4b and e we show slices of the manifold. In b they are evidence slices, where in and U2 are fixed and we evaluate o varying f, in order to obtain a trajectory of the dynamical system. In e we take time slices of the manifold by fixing t and evaluating for all U1 U2 0-2. 0.2]. The attractor planes were simply computed by letting t be large enough that the r.h.s. of the dynamical system became close to zero We note that this is not a "plane attractor" in the sense that the state of the system would tend to those values in the absence of inputs. They are manifold of attractor fixed points parameterised by #1, U2-
The metric. Under particular choice of coordinates - here the [t, u1, U2] coordinates - the metric at each point on the manifold is a x 3 positive (semi-)dee )definite symmetric matrix. In figure 4e we plot the metric at two time points on the manifold = 0.2 and t = 10 and for u1, U2 = 0. The results are nevertheless consistent over different values of u (Fig. S8).
Warping. In figure 4f and g we analyse the warping of the manifold across time and con- ditions. In panel f we compute the squared geodesic distance along either the un (top) or W2 (bottom) grid lines. For example for the top panels f ||0., (u u2) (d evaluated at E {-0.2,... .... -0.04, 0, 0.04, 0.2}. We divided this quantity by the total length of the geodesic to visualise only the effect of warping across values of the input as opposed to the overall stretch or compression of the manifold. Figure S8 shows the same analysis without this division, and integrated from -0. to μ (instead of to p), which further highlights the strength and context dependence of the warping. This is the same analysis performed on the deep neural network of figure 2, except repeated for the different values of the u that is not integrated over. In panel g we compute the eigenvalues of the metric. To be more precise, since the metric is a (0, 2)-tensor, the usual eigenvalue equation is meaningless and we instead compute the generalised eigenvalues of the metric via the equation Gv; X,/v, where is an identity metric. We average them over the irrelevant input, and
21

---

then normalised by the top eigenvalue at u1 = 0 in context 1 and d2 = 0 in context 2. The error bars are the cigenvalues over the irrelevant input. The fact that one eigenvalue is much larger at late time points suggests that space is strongly warped into a seemingly I 1-dimensional manifold. We note that this effect is less strong away from the decision boundary (Fig. S8), so the warping is not uniform over all relevant input values (as also shown in panel f).
Geometry causally affects computation. Here, we argue that the warping observed in figure is not simply correlative. To assess whether wrapping was necessary, we trained a network under the constraint that the diagonal elements of the metric remain of equal magnitude - meaning that the representation cannot compress irrelevant input. Conversely, to assess whether warping is sufficient, we trained a network simply to warp the irrelevant input dimension, before testing the model on the full task. Specifically, this was done by constraining the ratio of the diagonal elements of the metric to be either 1 or their value in the trained RNN. To do that, we defined new loss l:
(W, A, c (W, D) D) + 8(- Guats Meter
(c-
Gu G Mecto-1
where Grassteee Moca = 1|0 1l? and Meta is the input of the current context while U1-cta is the input of the opposite context. When preventing warping, we train all model parameters simultaneously. When we train only on warping, we first train W to warp the representation (a = 0, 8 = 10-2, c = 50) before holding it fixed while training the rest of the model parameters on the original task (ax 1, 8 = 0).
Model
Test performance (MSE, mean + std)
Untrained model
0.968 + 0.708
Baseline trained model
a = 1, =0 a 1, 10-2 c=
0.001 = 0.002
No warping
0.239 + 0.257
Untrained W
0.234 + 0.259
Only warping-trained W
a 0, B 10-2 c=
0.001 0.001
Interestingly, warping seems necessary to perform the task, in that the model constrained not to warp does not converge to the optimal task performance. Furthermore, more surprisingly, simply training the recurrent weights of the model to warp the representation, without any task information, sufficient to reach a performance nearly as good as the baseline model trained on the actual task. Thus, warping is necessary and sufficient element of the computation of the solution to this task. This further suggests that the warping observed is not purely correlational but indced the way by which the model performs the task.
Pre-trsining
1-0.8
Context 1
Context 2
Raloort input mag thudo
0.2
0.0
3.2
Dg menic
Post-training
t=10
0.2
Eg Im
Conted
0.2
Time
Time
Time
Figure S8: The context-dependent warping emerges training and is consistent across inputs. a. Eigenspectrum of the weights illustrating that the network has few large (real) eigenvalues (circled in red). b. Geodesic gridlines from u, -0.2 in context i. The gridlines from the two contexts are overlapping to illustrate that the stretch/compression of the irrelevant input over time ceneext-dependent. e. Metric over all inputs at early and late time points. d. Eigenvalues of the metric over different relevant inputs. The error bars are over the irrelevant input.
22

---

S8.4 Sequential working memory
In section 5 we used common model of working memory" 13,14 that is described in the main text. We used the same parameters initialisation as in the contextual evidence integration RNN. The hyper-parameters were:
9
Ir
Iterations
Optimiser ODE solver
PID " rtol, atol
100
0.5
5x10-1
3000
Adam
Heun
10 10-2 10-3
Gaussian curvature. In figure 6c we showed the Gaussian curvature of the torus at a particular time point. In principle the curvature could be computed in similar way to the metric using adjoint dynamics or automatic differentiation. However such methods would be computationally inefficient, especially to compute the curvature over fine mesh on the torus. To efficiently compute the Gaussian curvature we used a finite difference method. We evaluated the torus using the ODE solver and fixed step sizes 0.01 and mesh of angles 0, (0, A0,... Δ0} where A 0.01 (i.e. a mesh size of 200) and computed first order accuracy finite difference estimates of the terms of Gauss's Theorem Egregium formula:
0g x= (x(t, + (0,02))A4 A0
0s,x = (x(t, 01 02 + A0)) 00
V |0b, x, x)
Basis of tangent space
G VTV
Metric
0a. G - (G(1, 01 + A0, 02) 02))
0e2 G (G(t, 02 + 20) G(t, A2
k1 = (Ou, 82 - 0o Go (2)222))2111 /2v det
l/2 (Ou, Go, 02 0og (,) G
Ou, 11 = (k(0) + A0, 02) - k(01,02))/44 02))/20
0g 22 (k:01, 02 + A0) k(01, 0
K- = (0u, 11 - 022k22/ / det G
Gaussian curvature 15
To illustrate the curvature, we found the points 07. 03 where the curvature was highest (resp. lowest)
and took piece of the torus 0g E 10-- 0.19T 8r + 0.19r). defined the matrix X = [x(t,, f2) for these values of 02 and 02. and projected X- - (X)e,,9 .2 on the PC space.
Plotting the tori. In figure 6b we plot tori at different time points. They are all color coded according 0j. We computed the principal components after mean-cen That is on X [x 01 = 0, A2 0), x(t, Ao, 0), x(t, 0, At). .... xt, 2n,, 2r)] where AO = 0.01 and then projected X - (X)), .2.
The metric. In figure 6d we show the metric over the whole torus where the colormap is normalised to the 10th and 90th quantile per entry of the metric. In figure 6e (top) we averaged the metric over the torus, while in figure 6e (bottom) The error bars showed the 10th and 90th quantile of that value across the points on the torus.
first computed 20,,0/(Go, 0, + Go,0,) before averaging.
Subspace similarity. In figure 6g computed the similarity between the 2D PC spaces computed at any pairs of time points. The similarity is defined via the nuclear norm (L,) ((2)=111/ where V(4) is the 2 x matrix of the two right singular vectors with largest singular values. This defines notion of average angle between the subspaces." 16
Higher- dimensioll torus. In figure 6h, we trained new RNN on the same task but now with three targets at three angles 01 02 03. In figure 6i we show the squared geodesic distance along the gridlines defined by fg for all i. They are computed by choosing reference angles (here (01, 02 02) (I,, n, x)) and computing /igh = 5 Ge.e, (01, 02 0z)d0, (and similarly T.9_ J Ge.e; (01, O2 0g), The 0, side of the prism shown is of length ( + L The grid shown on the prism is obtained by varying the bound of (say the first) integral and normalising by the total integral L right
The topology of working memory. In theorem $7.3 we show that the state of the RNN at a particular time point must lie on the same manifold as the inputs. However, this manifold can be immersed in
23

---

d
T=2.750
250
Dalay Out
Qutput 2
T=2.0
Output
1.0
0:3
rong dolay
10
Decodar aignment
e, R.
Subospace sim similritty
Time
01234607860
04
Shon
62
-0.5 0:00
0.0
Gauasian curvature
0
Time 10
02
0.1 2. 3.3.5.5.. 7.90
Time
Figure S9: Working memory dynamically changes representational geometry. Geodesic gridlines at two different time points (mid-delay and of output) from 0; = to and 2x. The top row are the 0i geodesics. Unlike the decision task, there is no stronger warping around particular point on the manifold, because there no decision boundary. of the delay period. The curvature is more uniform than late in the trial (Fig. 6), but non-zero. Subspace (2D PC space) similarity during the whole task. d. Decoder alignment to the basis of the tangent space under two different delays. It suggests that it is the go signal that cues this alignment.
Gaussian curvature at the beginning
the neural state space (c.g. intersecting itself), or quotiented (c.g. if there is periodicity). Here we show that must be exactly torus at any time point after the second input presentation, and that is embedded in the state space. For this it suffices to prove that x(t, 01, 02) is a homeomorphism in its arguments.
Proposition S8.1. Let a working memory RNN defined as in section 5, and suppose the RNN performs the task correctly, meaning that Do(x(t1 01, 02)) = [cos(O1), sin(01)] where t1 is the first output time (and similarly for the second output). Then throughout the delay and output period the state of the RNN lies
boyper-)torus embedded in the RNN states-space IR" at any time point.
We will prove this for 2-torus (i.e. 2 targets), the proof generalises naturally to arbitrary number of targets.
Proof. After the second input has been presented the system is a composition of autonomous flows: (sy), o opž spt, (x(lo, 0,, 02)) where to is the time point where the second input is removed, ( is the delay, /2 the duration of the go cue and 13 the time until the end of the trial. Each flow is autonomous and therefore time- and initial-state diffeomorphism by lemma S7.4. Furthermore, a composition of diffeomorphic flows diffcomorphic will be an embedded torus if it is an embedded torus at particular time point.
and in particular homeomorphic. Therefore, the manifold
In theorem S7.3 we have shown that xt,, 01, 022 (ty 01, 02)) where P is projection map and yt, 01, 02) lies on a manifold of the same topology as its input, here a torus. It therefore remains to prove that P is bijection from the torus to its image (i.e. injective).
x(1, 09 03). Then, by time-diffeomorphismn, x(11, 0 05) x(1,, 02, 02) where L is the first output time. We can now apply the decoder on each side: Do(x(t1, (T.))) = (o(x(t1, 02 02)). By assumption, the RNN performs the task correctly, so this equality holds if and only if 0 = 0.. By the same argument for the time of the second output 03 07 Thus by the time diffcomorphism. for all t, x(t, 0,, 0b) x(t, 03, 02) if and only if (0), (1, = (0f, 02). This shows that x is topological embedding.
Let (0), 03) (02, such that x(1, 01 0b)
S8.5 Parametric working memory
We trained a model on a parametric working memory task called the Romo task. 17.18 The RNN sequentially receives inputs of varying magnitude through the same input vector.
* W (xxx)= x + bu,losssos +
where 41, U2 E [0.5, 1.0]. The task the network is to output until the end of delay period, and then the sign of the difference between uj and U2 afterwards, encoded via one-hot vector (Fig. S S10a).
L(W, = ||Do(x(t))) + 57 [|Do(x(t)) [1 >ua L
24

---

where 7i 4.0 is the end of the delay, T2 = 4.5 and T3 = 7.0 the end of the trial. Here D E R and @ the tanh nonlinearity.
n
9
Iterations
Optimiser
ODE solver
dt
100
0.5
10-3
2000
Adam
Heun
10-2
We found that the dimensionality of the manifold expanded during the delay (Fig. S10b, bottom). Indeed, the metric became multiple of the identity matrix (Fig. S10b, top), suggesting that the input which was originally encoded along single vector b had now been stored along two locally orthogonal directions 0, x and 0, of similar magnitudes, on the manifold. However, by the time of the output period the manifold had collapsed to become intrinsically 1-dimensional (Fig. S S10b, bottom). Furthermore, the partial derivatives 0a, x and 0u,x x pointed in opposite directions (Fig. S S10b, top). This is consistent with the network collapsing the representation of the two inputs onto line representing the output. We can consider the pullback of the subtraction operation: suppose that the two sequential inputs have been mapped to orthogonal vectors VI, V2 IR" such that the neural representation is given by x(u1, u2) = di Vi + V2 E IR" and consider the subtraction operation sp I - IR given by (ui, u2) V2 uy Then the pullback metric is:
-1
G(u1, u2) Jh J [1, - [1,
which matches the form of metric of the manifold near the output time (Fig. S10b, top). The sign operation and the mapping to the one-hot encoded outputs can be performed via the decoder Dé((). consistent with previously reported partitions of task computations into linear and binarisation operations." 19
Inpec 1 Inpuat 2 Dalay Output
Iria
Trial
0,,
Input
Input
2
Imput
Inpul
intrinsic dim.
Maximum 2.0
0
1-
Target ourput
RNN outout
Minimum 1.0
intrinsic dim.
0
Time
Time
Time
Figure S10: Neural dimensionality expands store working memories before collapsing to task outputs. a. Romo task: the network receives two inputs sequentially and then has to output the one-hot encoded sign of the larger input. b. Intrinsic dimensionality of the manifold, as quantified by the stable rank (sRank) of the metric, that is ||l||_
|[ Blue is over the submanifold where the output is
[0, 1], i.e. U2 un. Green is over the submanifold where the target output is [1, O], meaning i u2. The error bars are the 0.2 and 0.8 quantiles.
S8.6 State-space model on brain-computer interface data
To illustrate the usefulness our method data applications we trained state-space model (SSM) on brain-computer interface data. The data were recorded from a human subject instructed to draw centre-out lines." 20 We found that a one-layer HiPPO (LegS) SSM21 of the form:
T
* = Ax + Bu(t),
L(A, B) = Σ [x((t,) (1.))l
Where x(1) is the state of the SSM and u(t) is the time-interpolated input and yl,) the target cursor position. Since the task is performed offline, defined the target cursor position y(t)
25

---

r(t, coos sin 0] where the radius is given by:
< Lonset
r(t)
lonset S 1 bollset
Ite (201
t boffset
where E [1.3] is the length of the line.
Ir
Iterations
Optimiser
ODE solver
dt
100
10-3
2000
Adam
Heun
10-2
For fixed line length, our theory predicts that after the movement onset the state of the SSM will lie on a manifold of cylindrical topology representing the time t and angular variables (Fig. Slla). To study how different line lengths affect the dynamics, Akima cubic splines to define a continuous in-line-length input u. (t) to the SSM. Computing the time-length metric reveals that, during movement, the time and length partial derivatives become strongly aligned, suggesting that to perform longer lines neural activity accelerates (Fig. S11b). Such accelerating neural trajectories have been previously observed in RNNs and neural data from timing 23
interpolated the neural data using
tasks.
We thus illustrate that computing the metric allows to directly probe for this effect.
b
[axxi | 10xx
lidhsl] Akkll-1apa 11ax
SSM ectiviy
cirection
|0,x|| -1 1xx
[8xx
PCB
PCY
Moverment onsat
Mid- movemmat
Post- moxement enown))
Lia kirth
Carsor
BiX
Lina langth (1)
90
Figure S11: SSM trajectory speed varies generate BCI cursor movements of different lengths. a. Top: SSM activity in PCA space for a short timespan after movement onset. Bottom: Decoded cursor position. b. Length-time metric. At movement onset the time and line length ( variables locally encoded in orthogonal directions, while during movement they are strongly aligned.
Supplementary References
1. Lee, J. M. and Lec, J. M., Smooth manifolds, Springer, 2003.
Hauser, M. and Ray, A. 2017, Principles of Riemannian Geometry in Neural Networks, Advances Neural Information Processing Systems, ed. by Guyon, I. et al., vol. 30.
3. Zangrando, E. et al. 2024, Neural rank collapse: Weight decay and small within-class variability yield low-rank bias, arXiv preprint arXiv:2402.03991.
4. Aubry, M. and Russell, B. C. 2015, Understanding deep features with computer-generate imagery, Proceedings of the IEEE international conference computer vision, pp. 2875-2883
5. Hentschel. H. G. E. and Procaccia. I. 1983, The infinite number of generalized dimensions of fractals and strange attractors. Physica D: Nonlinear Phenomena 8(3), 435-444 444.
6. Kunita, H. and Ghosh, M., Lectures stochastic flows and applications, vol. 78, Tata Institute of Funda- mental Research Bombay, 1986.
7. Chen, R. T. et al. 2018, Neural ordinary differential equations, Advances in neural information processing systems 31.
Kingma, D. P. 2014, Adam: A method for stochastic optimization, arXiv preprint urXiv:1412.66.
9. Wong, K.-F. and Wang, X.JJ 2006, A recurrent network mechanism of time integration in perceptual decisions, Journal of Neuroscience 26(4), 1314-1328.
10. Kidger, P. 2022, On neural differential equations, arXiv preprint arXiv:2202.02435.
26

---

11. Mante, V. et al. 2013, Context-dependent computation by recurrent dynamics in prefrontal cortex, nature 503(7474). 78-84.
12. DeepMind, The DeepMind JAX Ecosystem, 2020.
13. Stroud, J. P. et al. 2023, Optimal information loading into working memory explains dynamic coding in the prefrontal cortex. Proceedings of the National Academy of Sciences 120(48). e2307991 120.
14. Cueva, C. J. et al. 2021, Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes, arXiv preprint arXiv:2111 1tt::2111.0111 1.01275.
15. Pressley, A. N., Elementary differential geometry, Springer Science & Business Media, 2010.
16. Hitzer, E. 2013, Angles between subspaces, arXiv preprint arXiv:1306.11 1629.
17. Romo, R. et al. 1999, Neuronal correlates of parametric working memory in the prefrontal cortex, Nature 399(6735), 470-473.
18. Schuessler, F. et al. 2020, The interplay between randomness and structure during learning in RNNs, Advances in neural information processing systems 33, 13352-13362
19. Brandon, J. Chadwick, A.. and Pellegrino, A. 2025, Emergent Riemannian geometry over learning discrete computations continuous manifolds, arXiv preprint arXiv:2512.00116.
20. Willett F. R. et al. 2021, High-performance brain-to-text communication via handwriting, Nature 593(7858), 249-254.
21. Gu, A. et al. 2022, How train your hippo: State space models with generalized orthogonal basis projections, arXiv preprint arXiv:2206.122 12037.
22. Akima, H. 1970, A method of interpolation and smooth fitting based local procedures, Journal of the ACM (JACM) 17(4), 589-602.
23. Wang, J. al. 2018, Flexible timing by temporal scaling of cortical responses, Nature neuroscicnce 21(1),
102-110.
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our contributions are clearly listed in the introduction. The supporting results can be found in: section 2 for the definition the representational metric and its solution via adjoint dynamics, section 3 the topological results relating the manifold the trajectories a dynamical system lie on to the topology of its input manifold and 4-5 several applications to example dynamical systems highlighting the insightfulness of this metric.
Guidelines:
The answer NA means that the abstract and introduction do include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In section we discuss both the limitations of our results and the broader limitations of the geometric perspective on dynamical systems.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model sell-specification,, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
28

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Several theorems presented in sections and 3. The full proofs including their assumptions are provided in supplementary material.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the information necessary reproduce the results are detailed in the supplementary materials S8.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution is a specific model and empirical evaluation, may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is either way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset instructions for how to construct the dataset).
model (e.g., .... large language model), then there should
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the
29

---

case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code necessary to reproduce all the figures is provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/ (oollic/guids/Coddeeee
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips.cc//pp cttppblic///cccccccccccccccc.llt http//ggddd//deeiieeeeeeeee
ssi ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only a subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
The core model used in each analysis are provided in the main manuscript, while all the details necessary to train the models, including all hyperparameters are provided in supplementary materials S8.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars are provided for all the metrics used to analyse the network.
Guidelines:
The answer NA means that the paper does not include experiments.
30

---

The authors should answer "Yes" if the results are accompanied by enror bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper discloses the compute resources used in supplementary materials (a single RTX 4070).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : http://cccc//ccccccccceeeeeee
Answer: [Yes]
Justification: We have read the NeurlPS Code of Ethics and confirm that the paper follows it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (c.g.. if there is special considera- tion due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
31

---

Justification: Our study does not have direct applications beyond the advancement of our understanding of biological and artificial neural networks
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, not needed point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No data was collected analysed part of this work and the models proposed do not have direct use beyond modelling the brain.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: No data was collected or analysed as part of this work, we wrote the code of all models.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
32

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ /nddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No data was collected or analysed part of this work.
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/codd/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No data was collected or analysed as part of this work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No data was collected or analysed part of this work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
33

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, or non-standard components.
Please refer to our LLM policy (https: lattps://neurips /ttp///rrrppecceooooooeeeeeeee/e (LMM for what should or should not be described.
34


Reviews:
[{"review_id": "5vgNi3owBP", "reviewer": "Reviewer_3Hky", "summary": "This work studies the geometry of the neural manifold and how it is shaped by a low-dimensional input manifold in an RNN dynamically. Specifically, the authors define a metric on the manifold that quantifies its shape and the evolution of its geometry across time. There are two main conclusions: contextual inputs warp (stretch) this manifold over time to disregard irrelevant inputs. Second, in the working memory task, the torus manifold is warped across time in the encoding phase, but it remains the same during delay.", "questions": "It would be nice to have some more intuition about what the metric represents. \n\nIn Fig.4f, representing the warping of the manifold, is it an illustration or an actual calculation of the geodesic gridlines?\n\nWorking memory is not just about maintenance but also about the manipulation of neural activity. How would you think the manifold would evolve, and how would your conclusions change if you considered a task with manipulation?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**: \n\nThe paper is very well written and is backed by mathematical proofs for the arguments made. The problem considered is also an important one: studying how neural manifolds evolve over time. The visuals in the figures make the point go across better!\n\n\n**Weaknesses**: \n\nThe main theorem about the outputs of RNN being low-dimensional if the input is low-dimensional seems trivial. Can the authors state if there are any special dynamical systems where this will not be satisfied, and/or explain why this is non-trivial?\n\nThe tasks considered in this work are low-dimensional, which might not be true for RNNs designed for ML. I understand the neuroscience motivation behind considering these tasks, but it would be interesting to see if this warping occurs in some task where the manifold is higher dimensional. It might not be feasible here, but a discussion about whether the conclusions drawn here would hold for a higher-dimensional RNN doing something more complicated, or state what the limitations of this approach are.\n\nAlthough there’s a calculation about the curvature of the torus but it seems no insights were drawn from it. So I’m not entirely sure of what the utility of that calculation is.", "was_revised": true, "final_justification": "The authors have provided a thorough and convincing rebuttal that addresses all the weaknesses I raised in my initial review. I recommend the paper for acceptance.", "rebuttal": "**Summary.** We thank the reviewer for the helpful and supportive comments. We were particularly pleased that the reviewer found the work to be “very well written” and “backed by mathematical proofs” while “the problem considered is an important one”.\n\nThe main comments raised by the reviewer were: \n\n* Questions regarding the applicability of the framework to higher-dimensional tasks and more complex models  \n* A request to provide better intuition regarding the meaning of the metric and manifold dimensionality   \n* The effect of manipulations of working memories on warping\n\nWe have addressed these points by:\n\n* Adding an example application where we find warping in an SSM trained on a BCI task  \n* i) improving the writing of the theoretical results section, ii) revising the schematic of figure 3, iii) moving the supplementary materials results regarding manifold topology to the main manuscript  \n* Adding an example of a common neuroscience task which requires computing the difference of numbers stored in working memory\n\nThe reviewer will find below our point-by-point responses.\n\n___\n**Results regarding the dimensionality of the RNN manifold.** Theorem 3.1 introduces a bound on the dimensionality of the manifold of states a dynamical system can take, as a function of the dimensionality of its inputs. We emphasise that this result is novel and non-trivial because 1\\) we consider the intrinsic dimensionality of the entire manifold of reachable states of the system, including transients far from steady state 2\\) we relate this dimensionality to the dimensionality of the input manifold in the space of functions, rather than input state space. \n\nMoreover, this theorem is in fact a corollary of a stronger theorem we derived in supplementary material (Theorem S7.3 and Corollary S7.5) that relates the topology of the manifold of inputs to the topology of the manifold of states of the dynamical system. In particular, these are related by a projection operator, which places strong constraints on the possible topologies the dynamical system state can take.\n\n*We thank the reviewer for pointing out that, as presented, the dimensionality theorem may seem trivial; we have therefore i) moved the topology theorem back in the main manuscript, and ii) mentioned the dimensionality result as a corollary.*\n\n**Special cases where this fails.** A key insight here is that the manifold of states to which the dynamical system is constrained is determined by the dimensionality of the manifold of time-varying input functions to the system. This contrasts with previous work which had focused on the dimensionality of the inputs *in state space at particular time points*, where this relationship to the dynamical systems’ dimensionality generally doesn’t hold. For example, there exists dynamical systems for which a 1D input can stir a system into any possible state (so-called “controllable systems”). \n\nThis also highlights a way in which the theorem can fail: if the assumption that we evaluate the system for a finite time is violated. In general, one can design a dynamical system and a single input function such that in the limit of large times the trajectory of the system is a space-filling curve. This means that although for finite time a dynamical system receiving a single input has solution a 1D manifold — its trajectory over time — in infinite time its trajectory can fill a dense region of space.\n\n___\n\n**Applicability of the framework to more complex tasks.** We thank the reviewer for asking whether the framework can be applied to tasks with higher-dimensional inputs and systems. Our framework is in principle applicable to higher-dimensional tasks and other dynamical systems typically used in ML (e.g. SSMs, NeuralODEs). The key is to define an insightful manifold of input functions to study.\n\nPrevious work has used the pullback of the metric to characterise the geometry of hidden layer activation of deep networks \\[1\\], for example networks trained on large image datasets (e.g. AlexNet on ImageNet) \\[2\\]. Despite the set of images living on a relatively high-dimensional manifold, these studies focused on the representation of low-dimensional manifold of possible orientations and positions of a fixed object in the image — which is a slice of the full “manifold of all images”. Similar approaches have since been applied to LLMs \\[3\\] and other architecture/tasks. \n\n*Our key theoretical contribution is to provide a non-trivial generalisation of this Riemannian geometric view on neural representations to dynamical systems. In general, one could take a dynamical system trained on large language, audio or video time series data, and study the geometry of the representation of a particular manifold of time-varying inputs to the system.*\n\n**New SSM model on a BCI task.** To address the important question raised by the reviewer regarding applicability to more complex tasks, we added an additional model of a state-space model (SSM) trained on a brain-computer interface (BCI) task. We chose this BCI task to remain within the neuroscience theme of the paper while having a system with higher-dimensional input data time series (brain recordings). \n\nWe based our model on a simplified version of a recent SSM architecture for neural data \\[4\\]. The model receives 256-dimensional human neural data time series which it must map to the velocity of a cursor on a screen. Because it is central to BCI control, we chose to study how the manifold of possible instantaneous velocities are encoded by the SSM during the task. In particular, in part of this dataset, the human subject is instructed to draw lines in different directions at different speeds from the center of the screen. This is a $S^1 \\\\times R$ manifold representing the possible orientations and speeds of the cursor. \n\n*Using our framework on this SSM trained on a BCI task, we found that neural activity warps along the time dimension across different velocities. In other words, to produce faster cursor movements neural activity speeds up along the trajectory in the neural state space.*\n\n___\n**Insights gained from the curvature of the torus.** We thank the reviewer for bringing to our attention the fact that the current writing doesn’t adequately motivate the analysis of the curvature of the torus. The motivation for this analysis was the long line of work suggesting that working memories are encoded in orthogonal spaces of neural activity space, which implies a flat manifold (zero curvature). Our analysis suggests that, at least in this classic RNN model of working memory, this view is inadequate: the torus is curved, and dynamically warps across the task, requiring a time-resolved analysis of transient states to be fully characterised, as per our method. \n\n*We have now better motivated the analysis of the curvature of the torus by more thoroughly discussing the existing literature on orthogonal representations of multiple items in working memory.*\n\n___\n**Intuition about the meaning of the metric.** The Riemannian metric characterises the intrinsic geometry of the manifold over which the state of the RNN lies. The representational “pullback” metric is defined in terms of the dot products of the partial derivatives $\\\\partial\\_{u\\_i}  \\\\mathbf{x}$ and $\\\\partial\\_{t} \\\\mathbf{x}$ which describe how the state of the system changes when the input function or time changes within the manifold of input functions. \n\nThe off-diagonal term can be interpreted as representing how correlated the encoding of the inputs are: if $G\\_{u_1,u_2}=$$\\\\partial\\_{u_1}  \\\\mathbf{x}$ $\\\\cdot$ $\\\\partial\\_{u_2}  \\\\mathbf{x} \\= 0 $ this means that the two variables are have an orthogonal encoding. \n\nThe diagonal terms can be be interpreted as the degree of space warping: in the extreme case where $G\\_{u_1,u_1}=||\\\\partial\\_{u_1} \\mathbf{x}||^2=0$ (meaning that the state of the dynamical system doesn’t change with this input), the state manifold is warped (compressed) such that the corresponding dimension of the manifold disappears (has zero volume).\n\n*To give the reader better intuition about the meaning of the metric, we have now included this more intuitive discussion, and revised the schematic from figure 3\\.*\n\n___\n**Is Fig.4f the actual warping?** We confirm to the reviewer that it is indeed the actual calculation of the geodesics. We were particularly pleased that the result so clearly highlighted that space was stretched around the decision boundary near the decision time. \n\n___\n**Effect of manipulation of working memories.** This is a great point raised by the reviewer. In our task, all the network has to do is to store in memory and then retrieve the inputs it is presented. Yet, there are several other neuroscience tasks — and real-world scenarios — which involve the manipulation of those memories (to perform calculations or else). \n\n*To better highlight the effect of manipulations of working memory we have now applied our framework to an RNN trained on a new task, where a network must perform a subtraction over numbers stored in memory \\[5\\]. We found that the manifold starts two dimensional (to represent both numbers in memory), and then dynamically warps and collapses to have a nearly rank-1 metric (i.e. being effectively 1-dimensional), to encode the difference of these numbers.*\n\n___\n\n**References**\n\n\\[1\\] Hauser, M., & Ray, A. (2017). Principles of Riemannian geometry in neural networks. *NeurIPS*.\n\n\\[2\\] Kaul, P., & Lall, B. (2019). Riemannian curvature of deep neural networks. *IEEE*.\n\n\\[3\\] Li, K. et al. (2023). Emergent world representations: Exploring a sequence model trained on a synthetic task. *ICLR*.\n\n\\[4\\] Ryoo, A. H. W. et al. (2025). Generalizable, real-time neural decoding with hybrid state-space models. *NeurIPS*.\n\n\\[5\\] Schuessler, F. et al. (2020). The interplay between randomness and structure during learning in RNNs.  *NeurIPS*."}, {"review_id": "1GqdXHrgbA", "reviewer": "Reviewer_RgJi", "summary": "In this paper authors use the framework of Riemannian geometry to analyse how RNNs compute through their dynamics. Through their analyses they identify that the computations of RNNs can be captured through the warping of their representational spaces. Authors trace this warping across a couple of established examples of RNN computations.", "questions": "Framework definition: In the abstract authors write \"To test this hypothesis, we develop a Riemannian geometric framework\" -- please comment on what you mean with developing a framework, given that Riemannian geometry has been used already. What specific organizing principles, predictive capabilities, or systematic methodologies does your framework provide beyond applying existing mathematical tools? This links to the criteria of clarity and originality. \n\nMetric definition and consistency: Is your view of your work that you specifically introduce a metric for warping that can be applied across examples? As stated in \"We introduce the representational metric defined as the pullback of the metric on the neural space to the input manifold\". I am struggling to see where you define a metric which is then coherently used across examples. Please clarify the definition of the metric and explain how this metric highlights mechanisms of computations in later examples. This links to the criteria of clarity and quality. \n\nMechanistic insights: My reading is that your investigations did not directly reveal any new mechanism about computations in dynamics but more focuses on the introduction of a metric? Please correct me if you have a specific theoretical result that I am overlooking. This links to significance. \n\nOverall contributions: Please provide an overall conclusion statement of your learning across investigations at the start of the 'Conclusions' section. Currently you only mention how Riemannian geometry can be applied across use cases, but it is unclear to me how your work contributes to this application of Riemannian geometry beyond demonstrating that existing mathematical tools can be applied to RNNs. This links to clarity and significance.  \n\nAdditionally, note typo in line 165: “neurosciencce”.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths \n\nAttempt to unify established examples of computations of RNNs through a coherent perspective of manifold warping \n\nStrong theoretical analyses of the expected dimensionality of RNN signals as a function of input signals \n\nSystematic application of differential geometric tools to RNN analysis \n\nBeautiful presentation of your results.\n\nWeaknesses \n\nWhy is this a new framework?: Authors claim to introduce a new framework but fail to explain in detail how their usage of Riemannian geometry goes beyond existing approaches for analysing dynamic representations.  \n\nInconsistent metric definition: The specific metric they develop based on Riemannian geometry claims to be based on pullback construction, but they fail to define or consistently apply a single coherent metric across analyses. Different sections seem to use different mathematical objects all labeled as 'representational metrics' without clear relationships between formulations. This plays into a larger issue that each of the individual sections seem to have substance by themselves but I am struggling to identify the links between them. This is not helped by the fact that authors do not provide a joint summary of findings across sections at the start of the discussion. \n\nWhat new findings are possible with the new framework: While perhaps the authors outline a new way of thinking about dynamic representations, they seem to not be able to use this analysis framework to derive findings which have not been reported in the literature yet. The claimed 'dynamic warping' phenomena mimics established findings about RNN learning dynamics (e.g., emergence of structured attractors, dimensionality reduction during training) using different mathematical vocabulary, without providing new computational insights. For discussion of toroidal representation there are existing works (https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011852, https://www.nature.com/articles/s41583-023-00693-x). For discussion of collapse of information over non-relevant dimensions, this is already shown in Sussillo's work and also in https://www.biorxiv.org/content/10.1101/2023.04.24.538054v2.abstract. In addition, Riemannian geometry has been used already for analysing (static) representations in https://proceedings.neurips.cc/paper_files/paper/2022/hash/4b3cc0d1c897ebcf71aca92a4a26ac83-Abstract-Conference.html.", "was_revised": true, "final_justification": "This work attempts to unify established examples of computations of RNNs through a coherent perspective of manifold warping. Its an interesting and well done body of work and we thank the authors for engaging during the discussion phase.", "rebuttal": "**Summary.** We thank the reviewer for their insightful comments. We are particularly pleased that the reviewer found our work to have “strong theoretical analyses” and “systematic applications of differential geometry” that we used to “unify established examples of computations of RNNs through a coherent perspective of manifold warping.”\n\nThe reviewer raised several questions regarding:\n\n* The relationship between our framework and i) other applications of Riemannian geometry in deep learning and ii) attractor manifolds in dynamical systems  \n* What insights about the computation can be gained by studying the geometry of neural networks\n\nWe’ve addressed these questions by:\n\n* Improving our introduction section to better highlight our contributions and how it relates to existing literature  \n* Performing a new set of validations where we explicitly manipulate the intrinsic geometry of the RNN representation to highlight that warping can be a necessary and sufficient component of task computation\n\nThe reviewer will find our point-by-point response below.\n\n___\n**Link to existing literature.** We thank the reviewer for the additional pointers to the literature. We confirm that neither the methods developed in our work, nor the results we obtained with them, had been previously reported. In particular, our work goes beyond previous studies by characterising the (time-varying) geometry of the manifold of states reached by the system upon all possible (time-varying) task inputs.\n\nWe have now cited in our paper the works highlighted by the reviewer and discussed their relationship to our own. We discuss this point-by-point below.\n\n**Link attractor manifolds.** The review mentioned by the reviewer (Langdon et al., 2024, *Nat. Rev. Neurosci.*) primarily discusses fixed point attractor manifolds, and the article on toroidal representation (Matthijs et al., 2024, *PLoS CB*) discusses limit cycles. Because they analyse steady-states and autonomous dynamics, these studies cannot directly account for time-varying inputs as our work can. \n\n🠢 More generally, methods for studying attractor manifolds can only infer a static manifold geometry, and therefore could not be used to tackle the main claim of our paper that representations dynamically change as task computations unfold.\n\nSussillo’s work (Mante et al. 2013, *Nature*) characterises variability, at steady-state, in the linearised dynamical system, near a line attractor encoding the context-relevant input. They find that the state of the system in response to pulse inputs decays for context-irrelevant but not relevant inputs. \n\n🠢 Instead, we i) analyse the full nonlinear system ii) characterise the geometry through the entire duration of the task (including the initial transient far from the line attractor), showing that the representation dynamically warps and, iii) characterise the intrinsic geometry of the manifold, showing both warping of the irrelevant input encoding and warping of the relevant input around its decision boundary. \n\n**Link to collapse.** Michał et al. (2024, *biorXiv*) discusses how deep neural networks with static inputs can collapse their state-space. Instead, we describe how *dynamical systems* receiving *time-varying inputs* can dynamically warp the *intrinsic geometry* of their manifolds. \n\nYaras et al., (2022, *NeurIPS*) studies the intrinsic geometry of neural representation of spheres in deep neural networks. They attempt to better understand how certain classes can be ignored during classification tasks via the so-called neural collapse phenomenon. \n\n🠢 Instead, we discuss how dynamical warping of the representation of time-varying inputs can be used by a dynamical system to perform computations. \n\n*Thus, although the works mentioned by the reviewer provide useful context to our own — in that they are part of the literature on using geometry to study neural networks — they do not significantly overlap with ours, neither in the mathematical tools they use, nor in the models they study.*\n\n___\n**Broader novelty of the framework.** Previous work has suggested that the pullback of the metric from the hidden layer representation of a neural network to the input manifold can provide insights into the computations it performs \\[1\\]. Such geometric approaches towards understanding neural computations have seen success in several applications ranging from convolutional networks in vision tasks \\[2\\] to transformers in language tasks \\[3\\].\n\nOur framework extends this view in that we develop a framework to study the geometry and topology of *dynamical systems*, described as differential equations, and which receive time-varying inputs on a manifold.\n\n* We prove a link between the topology of a manifold of time-varying inputs and the topology of the manifold over which the state of the system lies (Theorems 3.1, S7.3 and Corollary S7.5).   \n* We exactly derive the geometry of this manifold, defined as the pullback of the metric from the state-space of the dynamical system to the input manifold as a system of adjoint differential equations (Theorems 3.3, 3.4; Section S7.3).\n\n*To better address the link between our work and existing literature, we have now expanded on the related works section to better highlight how our work relates to other uses of Riemannian geometry in machine learning and computational neuroscience.*\n\n___\n**Metric definition and consistency.** We confirm to the reviewer that we are using the same metric throughout the work, with the minor caveat that in Fig. 6e we consider the metric of the slice of the manifold at a particular time point (a 2x2 submatrix of the 3x3 matrix of Fig. 6d), and in the rest of the work we consider the full space-time metric of the activity manifold. A general overview of the definition of the metric is presented in section 2, and more specific insights into how this metric is computed for deep neural networks and dynamical systems’ can be found in sections S7.1 and S7.2 of supplementary material. \n\nWe stress that this metric defines *the* intrinsic geometry of the neural manifold as it stands in the dynamical system’s state-space, and is not an object we’ve arbitrarily chosen. Other metrics can be placed on the manifold, but they would not match the geometry of the neural manifold within the state-space. \n\n*To better highlight this fact, we have now i) clarified the definition of the metric in the background section, and ii) moved some of the Riemannian geometry background from supplementary material to the main manuscript.*\n\n___\n**Mechanistic insights.** We acknowledge the reviewer’s comment that our analyses were purely correlative, in that we observe geometries that are linked to particular task variables or computations. For example, in our contextual decision-making RNN, we observe that i) the representation of the context-irrelevant input is compressed, and ii) near the decision boundary of the relevant input space is stretched, corresponding to the network switching from an output of $-1$ to $1$. Yet, there could be other mechanisms to solve the task that do not require these warping phenomena, or that the warping phenomena in themselves do not fully capture the computation performed by the network.\n\nTo further highlight that **warping is not an epiphenomenon** but a fundamental mechanism by which networks perform the tasks, we’ve now performed the following analyses in the contextual decision-making RNN:\n\n* To assess whether wrapping was necessary, we’ve now trained a network under the constraint that the diagonal elements of the metric remain of equal magnitude — meaning that the representation cannot compress an irrelevant input.   \n* To assess whether warping is sufficient, we’ve now trained a network simply to warp the irrelevant input dimension, before testing the model on the full task.\n\nWe’ve done this analysis by constraining the ratio of the diagonal elements of the metric to be either 1 or their value in the trained RNN. To do that, we’ve added a term to the loss: $\\lVert c - G_{u_{ctx}u_{ctx}}/G_{u_{ctx-1}u_{ctx-1}}\\rVert$ where $G_{u_{ctx}u_{ctx}}=\\lVert \\partial_{u_{ctx}} x\\rVert^2$ and $u_{ctx}$ is the input of the current context while $u_{ctx-1}$ is the input of the opposite context..\n\n|  | Untrained model | Baseline trained model | No warping | Untrained $W$ | Warping- trained $W$ |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| Test performance (MSE, mean $\\\\pm$ std) | $0.968 \\\\pm 0.708$ | $0.001 \\\\pm 0.002$ | $0.239 \\\\pm 0.257$ | $0.234 \\\\pm 0.259$ | $0.001 \\\\pm 0.001$ |\n\nInterestingly warping seems necessary to perform the task, in that the model constrained not to warp does not converge to a good task performance. Furthermore, more surprisingly, simply training the model on warping is sufficient to reach a performance nearly as good as the baseline model trained on the actual task. This suggests that **warping is a necessary and sufficient element of the computation** of the solution to this task.\n\n___\n\n**Discussion and broader impacts.** As requested by the reviewer, we have now provided an overall conclusion statement. We summarise and link together results from different sections. We highlight that we have developed new mathematical tools, which extend previous applications of Riemannian geometry in machine learning to the case of dynamical systems receiving time-varying inputs on a low-dimensional manifold. Furthermore, we explain that, using these tools, we can show that warping seems to be a ubiquitous feature of the computations performed by RNNs during classic neuroscience tasks.\n\n___\n\n**References**\n\n\\[1\\] Hauser, M., & Ray, A. (2017). Principles of Riemannian geometry in neural networks. *NeurIPS*.\n\n\\[2\\] Kaul, P., & Lall, B. (2019). Riemannian curvature of deep neural networks. *IEEE*.\n\n\\[3\\] Li, K. et al. (2023). Emergent world representations: Exploring a sequence model trained on a synthetic task. *ICLR*."}, {"review_id": "D8bq27bScD", "reviewer": "Reviewer_aFgg", "summary": "The authors present a formal framework for understanding computation through dynamics present in recurrent neural network architectures with time-varying inputs. They study the dimensionality and topology of network dynamics with respect to the structure of the input data for both contextual decision making and working memory tasks. On simple models of a contextual classification and memory task, they demonstrate how the geometry of the dynamics is warped to allow the network to discard (contextually defined) irrelevant information through dynamics.", "questions": "- In section 2, does the choice of Tanh non-linearity have a significant effect on the warping of the space near the decision boundary compared to a ReLU? More generally, can the authors comment on how the choice of piece-wise linear non-linearity will impact their results compared with 'sigmoid-like' non-linearities? \n- It appears a proof of Theorem 3.4 is lacking from the supplementary material. Is this somehow given by 3.3? Should it then be written as a corollary?\n- Can the authors comment on the relationship of their theory to that of Low-Rank RNNS? Adding a discussion of this to the paper would be beneficial for the community.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The formal study of time-varying inputs in RNN dynamical systems is indeed something which is lacking in the literature, and a valuable contribution, especially to the computational neuroscience community (which the authors appear well aware of). \n- The framework seems broadly applicable to the study of nonlinear dynamics, and therefore has the potential to yield novel insights not possible with current methods.\n- The paper is well written and organized for readability and understanding. For example, the review of core concepts for static neural representations in Section 2 is a welcome addition.\n- The proposed framework and viewpoint for understanding computation in recurrent neural networks is novel to the best of my knowledge and appears to open many doors for further understanding how RNNs perform computation -- the authors also clearly demonstrate this potential through their chosen examples.\n\n\n**Weaknesses**\n- The paper seems to have a relatively significant connection to the recent work on Low-Rank RNNs from Valente, Pillow & Ostojic, (Neurips, 2022). The authors cite this paper, yet have no further discussion on the relationship to that theory.", "was_revised": true, "final_justification": "The paper is well written, technically sound, creative, and a novel method for understanding computation in RNNs. The authors have addressed all my concerns regarding the relation to prior work with exceptional clarity, and have provided additional experiments. I believe the paper is a clear benefit to the community and would make a strong contribution to the conference.", "rebuttal": "**Summary.** We thank the reviewer for his positive comments. We are particularly pleased that he found our work to be “broadly applicable to the study of nonlinear dynamics” such that it has “the potential to yield novel insights not possible with current methods”, and that he generally found it to be “well written.”\n\nThe reviewer raised some questions regarding:\n\n* The relationship between our work and the recent work on low-rank RNNs (e.g. Valente, Pillow, Ostojic, 2022, *NeurIPS*)  \n* The effect of the choice of nonlinearity on the geometry (especially in the model from section 2\\)\n\nWe’ve now addressed these questions by:\n\n* Improving our discussion and background sections to better relate our work to the low-rank RNN framework  \n* Providing additional simulations showing how various nonlinearities affect the intrinsic geometry.\n\nWe thank the reviewer for very positively receiving our work, and we believe that the revisions stemming from his comments have helped further highlight the novelty and generality of our work. \n\nWe have provided below point-by-point responses to the reviewers questions.\n\n___\n**Relationship to the low-rank RNN framework.** We thank the reviewer for raising this point, and agree that a deeper discussion of the relationship to work on low-rank RNNs will be of substantial benefit to readers in the computational neuroscience community. The key distinction between our work and low-rank RNNs is that low-rank RNNs restrict the *embedding* dimensionality of RNN activity, while our work mathematically characterises the *intrinsic* dimensionality.\n\nIndeed, if the connectivity of an RNN is low rank, meaning that $W=\\\\sum_{i=1}^r \\\\mathbf{u}_i\\\\mathbf{v}_i^T$, then the dynamics of the RNN upon receiving a d-dimensional input $\\\\mathbf{a}(t)$ can be written as \n\n$$\\\\frac{d\\mathbf{x}}{dt}=-\\mathbf{x}+W\\\\phi(\\mathbf{x})+B\\\\mathbf{a}(t)=-\\mathbf{x}+\\\\sum_{i=1}^r \\\\mathbf{u}_i (\\\\phi(\\\\mathbf{x})\\\\cdot \\\\mathbf{v}_i) + \\\\sum_i^d \\\\mathbf{b}_i a_i(t)$$\n\nwhich is a linear combination of the $\\\\mathbf{u}$'s and $\\\\mathbf{b}$'s (plus a leak term). In particular, this means that the dynamics of low-rank RNNs are constrained to an $r+d$ dimensional subspace, with activity in all other dimensions rapidly decaying due to leak. Furthermore, this means that if the system is initialised within this subspace, the state of the low-rank RNN remains embedded in this $r+d$-dimensional subspace.\n\nCrucially, in low-rank RNNs the input $\\mathbf{a}(t)$ is $d$-dimensional at any particular time point. Instead, we constrain the inputs to the system to lie on an $m$-dimensional manifold of time-varying functions which can take on arbitrary values in n-dimensional state space. That is, each point on the manifold is a different function. In table 1 of our manuscript we show the $d$ and $m$ dimensionality of various models and tasks typically considered in computational neuroscience. In general, $d$ can be high while $m$ is low, and vice versa. In addition, we show that activity manifolds are low-dimensional even when connectivity is full-rank and inputs are high-dimensional in state space, provided that the inputs are constrained to a low-dimensional manifold of functions.\n\nThe two frameworks can be naturally united if we combine their assumptions, that is if:\n\n1. The connectivity matrix W is low-rank — of rank r   \n2. The instantaneous input $\\mathbf{a}(t)$ lies on a low-dimensional space — of dimension $d$   \n3. The input function $\\\\mathbf{a}$ lies on a low-dimensional manifold (in function space) — of dimension $m$.\n\nIn this case, by the results we derive (Theorem 3.1, S7.3, and corollary S7.5), the state of the system lies on a low-dimensional nonlinear manifold of dimension $m+1$, and via the theorems derived in Ostojic’s work \\[1-2\\] this manifold will be embedded within an $(r+d)$-dimensional subspace of the RNN state-space. In other words, the low-rank RNN framework places constraints on the linear embedding dimensionality of the state while our framework places constraints on the intrinsic dimensionality of the nonlinear manifold of states the RNN can take. \n\nWe believe that future work could take advantage of the natural link between these frameworks to design models that better match experimental data. For example, one may find — perhaps using dimensionality reduction methods — that recordings from a particular brain region during a particular task are embedded in a $d$-dimensional space, but also lie on a $m$-dimensional nonlinear manifold. Then, one could design an RNN model that matches these experimentally observed phenomena. This could be made possible by the extensive line of work characterising the intrinsic and embedding dimensionality of neural activity \\[3-4\\]. Future works could also attempt to understand how placing constraints on the embedding dimensionality by imposing low-rank connectivity affects the intrinsic geometry of the nonlinear manifold.\n\n*We think that this connection between the well-established low-rank RNN framework and our framework is a strength, which could help the reader gain intuition about how the manifolds we discussed are defined. We have therefore expanded on this link in the introduction and background sections, including additional references to the work we cite below, and a new schematic introducing the links between intrinsic and embedding dimensionality.*\n\n___\n**The effect of the choice of nonlinearity in section 2\\.** We agree with the reviewer that it is important to understand how model choices affect the geometry. We have now added a supplementary material figure where we re-train the model from section 2 with various nonlinearities (ReLU, Tanh, Softplus and GeLU). \n\nWe found that the intrinsic geometry of the manifold was very stable across nonlinearities. We’ve included below a table showing the difference in geodesic distance (which is a quantity between 0 and pi/2) between the baseline tanh model and these other models.\n\n|  | Tanh | Tanh (different seed) | Softplus | ReLU | GeLU | Tanh (pre-training) |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| Difference in geodesic distances from decision boundary $\\int_0^\\pi \\|\\gamma_{\\text{model 1}}(\\theta)-\\gamma_{\\text{model 2}}(\\theta)\\| d\\theta$ | 0 | $0.007 \\\\pm 0.007$ | $0.031 \\\\pm 0.042$ | $0.009 \\\\pm 0.013$ | $0.022 \\\\pm 0.031$ | $0.273 \\\\pm 0.229$ |\n\nWe interpret this as meaning that the warping is closely tied to the computation performed by the network: small changes in input angles near the decision boundary will correspond to large changes in the hidden layer activity, in turn corresponding to switching from an output of \\-1 to 1\\. Still, in more complex tasks, we expect that different networks could use fundamentally different solutions, which have different intrinsic geometries. \n\n___\n**Proof of theorem 3.4.** We thank the reviewer for catching this. The proof of theorem 3.4 directly follows from that of theorem 3.3 with one additional step of derivation. To make that clearer, we have now added this step as a separate proof in supplementary material, and as suggested by the reviewer, we have now relabelled this result as a corollary.\n\n**Overall.** We thank the reviewer for his positive and constructive comments which have helped improve the work. In particular, we believe that the discussion of the relationship between our work and the low-rank RNN framework will help explain our framework to a broader audience, and that the additional validations of the effect of model choices will help highlight warping as a key phenomenon in neural network computation.\n\n**References**\n\n\\[1\\] Valente, A., Pillow, J. W., & Ostojic, S. (2022). Extracting computational mechanisms from neural data using low-rank RNNs. Advances in Neural Information Processing Systems, 35, 24072-24086.\n\n\\[2\\] Mastrogiuseppe, F., & Ostojic, S. (2018). Linking connectivity, dynamics, and computations in low-rank recurrent neural networks. Neuron, 99(3), 609-623.\n\n\\[3\\] Beiran, M., Meirhaeghe, N., Sohn, H., Jazayeri, M., & Ostojic, S. (2023). Parametric control of flexible timing through low-dimensional neural manifolds. Neuron, 111(5), 739-753.\n\n\\[4\\] Jazayeri, M., & Ostojic, S. (2021). Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity. Current opinion in neurobiology, 70, 113-120."}, {"review_id": "ReAd0aVz9H", "reviewer": "Reviewer_2FTC", "summary": "The paper introduces a framework for understanding computation in recurrent neural networks (RNNs) based on Riemannian geometry. The central hypothesis is that RNNs perform computations by dynamically warping the geometry of their internal neural representations. The authors develop a mathematical formalism to test this hypothesis. First, they show that if the time-varying inputs to a dynamical system lie on an $m$-dimensional manifold, the trajectories of the state are constrained to a manifold of at most $m+1$ dimensions. Then, they define a \"representational metric\" on this neural activity manifold, derived as a pullback from the state space. This metric quantifies the local \"warping\" of the representation space and can be computed by solving an adjoint differential equation alongside the main RNN dynamics.\n\nThe framework is applied to two toy tasks: In the first task, the model has to output the sign of one of two real inputs, selected based on a (one-hot) contextual clue. The analysis reveals that the network learns to warp its representational manifold to selectively compress the dimension corresponding to the irrelevant input, while expanding the dimension for the relevant input near the decision boundary (close to zero). In the second task, the model has to reproduce two inputs after a time delay. The authors show that the network activity lies on a toroidal manifold, which dynamically changes its intrinsic geometry to selectively encode and recall items from memory.", "questions": "1. Have you considered applying your framework to RNNs trained on real-world tasks, for example, for natural language processing? I think it could be very insightful to explore how different architectures (e.g., LSTMs vs. SSMs) solve tasks. Maybe it would even be possible to extend the framework, so that it can be applied to (autoregressive) transformers? They can be considered as a special kind of RNN, where the size of the internal state (the KV-cache) grows over time (this paper https://arxiv.org/abs/2006.16236 is helpful to make this analogy clearer).\n2. Do you see a path toward using these geometric insights prescriptively? For instance, could the representational metric be incorporated into the training process (perhaps as a sort of regularizer), to encourage a model to learn specific geometric structures and potentially improve learning speed and/or generalization?\n3. Under Limitations, you mention the degeneracy of the map from network parameters to geometry. Have you investigated whether different parameter initializations converge to solutions with similar geometric properties? Perhaps more complicated tasks are necessary to observe \"distinct solutions\" (for the investigated toy tasks, it is difficult to think of different solutions).\n\nTypos (these do not influence the rating):\n\n- line 231: \"context-depenent\" should be \"context-dependent\"\n- caption of Figure 6b: \"is lies\" should be \"lies\"", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper is well-written and organized. Many helpful illustrations allow the reader to develop an intuitive understanding of the relevant concepts and ideas.\n2. The mathematical framework is rigorously developed and intuitively makes sense. It goes beyond an analysis of the dynamis near fixed points.\n\n**Weaknesses**\n1. The authors only apply their framework to very small models and simple toy tasks. While this is helpful to illustrate central concepts, an application to larger models trained to solve real-world tasks is lacking. Without this, it remains unclear whether the framework can be used to gain an understanding of the inner workings of more complex models.\n2. While the framework provides an account of how a model's internal representations are transformed, the link between geometry and computation seems correlational: While the geometry warps *as* the model performs the task, it is not evident to me that this geometric transformation *is* the computation itself, versus a by-product of another underlying process (i.e. the \"actual computation\"). The presented framework is currently post-hoc, and it is not yet clear if it can be used prescriptively to design or train models more effectively.\n\nWhile I find the general premise of the paper very interesting, my main concern (and why I did not rate it higher) is that it feels \"incomplete\" to me. The paper does not apply the developed framework to any \"real problem\" and does not demonstrate what kind of improvements or advancements it enables. I think the paper would be much more impactful if it demonstrated a few applications beyond toy examples.", "was_revised": true, "final_justification": "The authors addressed many of the weaknesses I raised in my original review. I still believe if the authors looked at \"more relevant\" tasks (e.g., language modelling), it would have made the paper potentially more interesting for a wider audience, but I respect the author's decision to stick to the neuroscience theme of the paper. I think the paper could be accepted, but will only be of interest for a relatively narrow audience (hence the borderline rating).", "rebuttal": "**Summary.** We thank the reviewer for his supportive comments and the overall feedback he provided on our work. We are particularly pleased that he found that “the mathematical framework is rigorously developed” going “beyond an analysis of the dynamics near fixed points”, while still \"developing an intuitive understanding.” \n\nThe main questions raised by the reviewer were:\n\n* Whether our framework could be applied to more complex models and tasks  \n* Whether studying the geometry of neural networks could provide insights into computational mechanisms, beyond correlations\n\nTo  address these comments we have now:\n\n* Added an application of our framework to state-space models (SSMs) trained on a brain-computer interface (BCI) task.  \n* Added an analysis where we explicitly manipulate the geometry of RNN activity to show that warping can be a necessary and sufficient component of task computation\n\nWe thank the review for these comments as we believe that these additional analyses have helped make the work more appealing to a broader audience, and that it now better highlights warping as a fundamental component of task computation in dynamical systems. \n\n___\n**Applications to more complex models.** We thank the reviewer for suggesting additional applications, and confirm that our Riemannian geometry framework is applicable to a large class of dynamical systems and tasks. We agree with the reviewer that an example network trained on real-world data would make the work more appealing to a broader audience. We have therefore added a **new application of our framework to state-space models trained on a brain-computer interface task**. \n\nThe SSM is trained to map high-dimensional time series data of brain activity to intended cursor velocities on a screen. We adapted an existing SSM architecture (POSSM \\[1\\]), to this task. We chose this task to provide an application to real-world data while still remaining within the computational neuroscience theme of the paper.\n\nBecause speeds and directions of the cursor are central to BCI tasks, we asked how the SSM model represented them. For this, after training the model, we looked at a subset of the time-series data, where the human subject systematically varied the intended heading angle and velocity of the cursor, starting at the center of the screen. Thus, the neural signals should approximately lie on the manifold $\\\\mathcal{M}=S^1 \\\\times \\\\mathbb{R}$, representing the direction and speeds of the cursor. This corresponds to a slice of the full manifold of all possible cursor movements. \n\nWe derived the representational metric of this manifold by deriving the pullback from the SSM state-space to $\\\\mathcal{M}$. Doing so revealed that different velocities correspond to different warpings of the neural manifold representing these velocities. We found that the non-diagonality was high, which suggested that the encoding of speed and time on the manifold were strongly correlated. In other words, to make cursor movements faster, the neural activity accelerates. This phenomenon had been previously reported in the neuroscience literature in other timing tasks \\[2\\]. This corresponds to a warping of the neural manifold along the time dimension as the speed increases.\n\nThus, our framework can be applied to more complex models trained on larger tasks. The key step is to define the manifold that one wishes to study — here velocities and directions. In general this manifold will be a submanifold of a larger manifold of inputs over which the model was trained. For example, previous work has tried understanding vision models trained on large-scale image classification, and where the manifold of interest was the position or orientation of the object in the image \\[3\\]. Furthermore, LLMs where the manifold of interest might be a constrained set of inputs for a fixed prompt \\[4\\]. \n\nThe paper mentioned by the reviewer linking transformers and RNNs — which we have now cited —  suggests that the tools we have developed could in principle be applied to those models.\n\n**“Toy-tasks”.** While we believe that the additional application to real world data suggested by the reviewer significantly improves our work, we note that the “toy-tasks” considered in this paper are well-studied and fundamentally important problems in neuroscience. For example, the contextual decision-making tasks we considered are used in the computational neuroscience \\[5\\] and machine learning \\[6-7\\] literature to probe the inner-workings of biological and artificial neural networks. These tasks, although simple, are not trivial. They require particular computations which the neural network — whether biological or artificial — must implement through specific patterns of neural activation. \n\nTo better highlight that the models presented, although simple, are tackling “real problems”, we have now discussed in the introductory paragraph to each section how the model and task we analyse have been previously used in the computational neuroscience and machine learning literature to understand dynamical neural computations.\n\n___\n**Insights beyond correlations.** We agree with the reviewer that our study is correlational: we observe that in multiple models, the representation is warped as the  task is performed, and that this warping correlates with the computations performed in the task. For example, in the contextual decision-making task, we found that the manifold is warped (compressed) along the context-irrelevant input. However this warping could in principle be a by-product of another more fundamental mechanism by which the model performs the task.\n\nTo further highlight that **warping is not an epiphenomenon** but the fundamental mechanisms by which the network performs the task, we’ve now performed the following analyses in the contextual decision-making RNN:\n\n* To assess whether wrapping was necessary, we’ve now trained a network under the constraint that the diagonal elements of the metric remain of equal magnitude — meaning that the representation cannot compress an irrelevant input.   \n* To assess whether warping is sufficient, we’ve now trained a network simply to warp the irrelevant input dimension, before testing the model on the full task.\n\nWe’ve done this analysis by constraining the ratio of the diagonal elements of the metric to be either 1 or their value in the trained RNN. To do that, we’ve added a term to the loss: $\\lVert c - G_{u_{ctx}u_{ctx}}/G_{u_{ctx-1}u_{ctx-1}}\\rVert$ where $G_{u_{ctx}u_{ctx}}=\\lVert \\partial_{u_{ctx}} x\\rVert^2$ and $u_{ctx}$ is the input of the current context while $u_{ctx-1}$ is the input of the opposite context.\n\n|  | Untrained model | Baseline trained model | No warping | Untrained $W$ | Only warping-trained $W$ |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| Test performance (MSE, mean $\\\\pm$ std) | $0.968 \\\\pm 0.708$ | $0.001 \\\\pm 0.002$ | $0.239 \\\\pm 0.257$ | $0.234 \\\\pm 0.259$ | $0.001 \\\\pm 0.001$ |\n\nInterestingly, warping seems necessary to perform the task, in that the model constrained not to warp does not converge to the optimal task performance. Furthermore, more surprisingly, simply training the model on warping is sufficient to reach a performance nearly as good as the baseline model trained on the actual task. Thus, **warping is a necessary and sufficient element of the computation** of the solution to this task. This further suggests that the warping observed is not purely correlational but indeed the way by which the model performs the task.\n\n\n___\n\n**Robustness of the metric across initialisations and model hyperparameters.** We have now added an analysis showing the robustness of the metric to different random seeds. The geodesic distance between points on the manifold is very similar across model initialisations. In the network from section 2, we find an expected geodesic distance difference of:  $E\\\\left\\[\\\\int\\_0^{2\\\\pi} \\|\\\\gamma\\_1(\\\\theta)-\\\\gamma\\_2(\\\\theta) \\|d\\\\theta\\\\right\\]=0.0227$.\n\nWe can nevertheless get distinct solutions by varying the network hyperparameters. For example, previous work has suggested that varying the magnitude of the initialisation can make the network transition between a rich learning regime where it learns a general solution to the task to a lazy regime where it overfits particular inputs \\[8\\]. We find that the intrinsic geometries under different initialisations are significantly different $E\\\\left\\[\\\\int\\_0^{2\\\\pi}\\| \\\\gamma\\_1(\\\\theta)-\\\\gamma\\_2(\\\\theta) \\|d\\\\theta\\\\right\\]=0.19498597$, close to the difference between a trained and random network $E\\\\left\\[\\\\int\\_0^{2\\\\pi} \\|\\\\gamma\\_1(\\\\theta)-\\\\gamma\\_2(\\\\theta) \\| d\\\\theta\\\\right\\]=0.273$.\n\nThus, while the metric is generally robust across seeds, one can design models with qualitatively different geometries (corresponding to different generalisation abilities) by varying model hyperparameters.\n\n___\n\n**References**\n\n\\[1\\] Ryoo, A. H. et al. (2025). Generalizable, real-time neural decoding with hybrid state-space models.*NeurIPS*\n\n\\[2\\] Beiran, M.et al (2023). Parametric control of flexible timing through low-dimensional neural manifolds. *Neuron*.\n\n\\[3\\] Kaul, P., & Lall, B. (2019). Riemannian curvature of deep neural networks. *IEEE*.\n\n\\[4\\] Li, K. et al. (2023). Emergent world representations: Exploring a sequence model trained on a synthetic task. *ICLR*.\n\n\\[5\\] Mante, V. et al. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. *Nature*.\n\n\\[6\\] Valente, A. et al. (2022). Extracting computational mechanisms from neural data using low-rank RNNs. *NeurIPS*.\n\n\\[7\\] McMahan, B. et al. (2021). Learning rule influences recurrent network representations but not attractor structure in decision-making tasks. *NeurIPS*.\n\n\\[8\\] Saxe, A. et al. (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. *arXiv*."}]

Author rebuttal:

Meta-review: This paper proposes a framework based on Riemann geometry to argue that RNNs perform computations by dynamically warping the manifolds of their internal representations. Reviewers found the work to be well-presented and mathematically sound but initially raised concerns about its novelty and the simplicity of the tasks. In rebuttal, the authors successfully addressed most technical critiques by clarifying their contributions and adding new experiments on a more complex BCI task and causal analyses of the warping mechanism. While these changes improved the completeness of the analysis, the continued narrow focus on neuroscience-themed problems may limit its impact on a broader audience. Nevertheless, the community may find the techniques and perspective interesting, and I recommend the paper for acceptance.
Decision: Accept (poster)