Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis
Authors: Mitchell Ostrow
Adam Joseph Eisen
Leo Kozachkov
Ila R Fiete
Venue: NeurIPS 2023 poster (2023)
Keywords: Computational Neuroscience
Neural Data Analysis
Statistical Shape Metrics
Representational Similarity Analysis
Recurrent Neural Networks
Dynamical Systems
TL;DR: We introduce a novel metric to compare the computations of two RNNs (or more generally, two systems that evolve over time) at the level of their dynamics.

Abstract: How can we tell whether two neural networks utilize the same internal processes for a particular computation? This question is pertinent for multiple subfields of neuroscience and machine learning, including neuroAI, mechanistic interpretability, and brain-machine interfaces. Standard approaches for comparing neural networks focus on the spatial geometry of latent states. Yet in recurrent networks, computations are implemented at the level of dynamics, and two networks performing the same computation with equivalent dynamics need not exhibit the same geometry. To bridge this gap, we introduce a novel similarity metric that compares two systems at the level of their dynamics, called Dynamical Similarity Analysis (DSA). Our method incorporates two components: Using recent advances in data-driven dynamical systems theory, we learn a high-dimensional linear system that accurately captures core features of the original nonlinear dynamics. Next, we compare different systems passed through this embedding using a novel extension of Procrustes Analysis that accounts for how vector fields change under orthogonal transformation. In four case studies, we demonstrate that our method disentangles conjugate and non-conjugate recurrent neural networks (RNNs), while geometric methods fall short. We additionally show that our method can distinguish learning rules in an unsupervised manner. Our method opens the door to comparative analyses of the essential temporal structure of computation in neural circuits.

Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis
Mitchell Ostrow, Adam Eisen, Leo Kozachkov, Ila Fiete Department of Brain and Cognitive Sciences, MIT
Cambridge, MA, USA
fostrow, [istron, a fiete] @mit..du
Abstract
How can we tell whether two neural networks utilize the same internal processes for particular computation? This question is pertinent for multiple subfields of neuroscience and machine learning, including neuroAI, mechanistic interpretability, and maii-m interfaces. Standard approaches for comparing neural networks focus on the spatial geometry of latent states. Yet in recurrent networks, compu- tations implemented at the level of dynamics, and two networks performing the same computation with equivalent dynamics need not exhibit the same ge- ometry. To bridge this gap, we introduce novel similarity metric that compares two systems at the level of their dynamics, called Dynamical Similarity Analy- sis (DSA). Our method incorporates two components: Using recent advances in data-driven dynamical systems theory, we learn high-dimensional linear system that accurately captures core features of the original nonlinear dynamics. Next, we compare different systems passed through this embedding using novel extension of Procrustes Analysis that accounts for how vector fields change under orthogonal transformation. In four case studies, we demonstrate that our method disentangles conjugate and non-conjugate recurrent neural networks (RNNs), while geometric methods fall short. We additionally show that our method can distinguish learning rules in an unsupervised manner. Our method opens the door to comparative analyses of the essential temporal structure of computation in neural circuits.
Introduction
Comparing neural responses between different systems or contexts plays many crucial roles in neu- roscience. These include comparing model to experimental data (as a measure of model quality,
Schrimpf et al. 22200), determining invariant states of neural circuit (Chaudhuri et al. (2019)) )
identifying whether two individual brains are performing computation in the same way, aligning recordings across days for a brain-machin interface (Degenhart et al.) [2020), and comparing two models (e.g. probe the similarity of two solutions to problem. Pagan et al. [2022]). )). Current similarity methods include R² via Linear Regression, Representational Similarity Analysis, Singu- lar Vector Canonical Correlation Analysis, Centered Kernel Alignment, and Procrustes Analysis (Schrimpt et al. 22020), Kriegeskorte et al. [2008), Raghu et al. [2017), Williams et al. 202T), J Duong
et al. 2022 ). Crucially, these all compare the geometry of states in the latent space of neural network.
Identifying shared neural computations between systems calls for methods that compare similarity at the level the fundamental computation. the brain, computations are instantiated by the emergent dynamical properties of neural circuits, such as fixed points, invariant and fixed point manifolds,
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

limit cycles, and transitions between them Amari and Arbib 1977 Hopfield [1984]. Hopfield and
Tank 1986], [Seung [1996]. Zhang 1996], Hahnloser et al. 2000 Burak and Fiete 2009 Wang
[2002]
Sussillo and Abbott 2009], Churchland et al. [012]. Chaudhuri et al. [2019]. Vyas et al.
[2020] Khona and Fiete
similarity metrics fall short in two manners when they
are applied to dynamical systems. First, response geometries can differ due to sampling differences
(Chaudhuri et al. (019)) or other nonfundamental causes even though the dynamics are topologically equivalent. In such cases, measures of state-space geometry do not fully capture the core similarities between two systems, a false negative situation. Conversely, systems may exhibit distinct dynamics over similar geometries in the state space (Galgali et al. (023)) ). In this case, geometric measures may fail to distinguish two distinct systems, a false positive situation. Therefore, we suggest that dynamical similarity should be the preferred lens through which to compare neural systems.
In dynamical systems, key notion of similarity is called topological conjugacy: the existence of a homeomorphism that maps flows of one system onto flows of the other. When two systems are conjugate, they have the same qualitative behavior. Given two dynamical systems f X - X and
g: Y + Y with invertible mapping & X Y. conjugacy is defined as:
Φ f
(I)
Here, we develop a data-driven method called Dynamical Similarity Analysis (DSA), which combines work from two previously unrellate fields, machine learning for dynamical systems and statistical shape analysis. In brief, DSA returns similarity metric describing how two systems compare the
level of their dynamics. First, DSA uses the insight that appropriate nigh-dimensional embeddings can be used describe a highly nonlinear system in a way that is globally linear. The features that enable this transformation are referred to as Koopman Modes (Mezić (2055) and can be identified via the Dynamic Mode Decomposition (DMD) (Snyder and Song [2021], Brunton et al. (2021)). Subsequently, DSA employs statistical shape analysis metric to compare vector fields within the Koopman Mode embeddings, thereby assessing the similarity between the systems dynamics. This shape analysis measures how far a pair of systems are from being homeomorphic. Unlike most spatial shape analyses, which require averaging over multiple noisy trials in the same condition to arrive at one vector for each condition-tine point (but see Duong et al. [2022), ) small amounts of noise create a rich representation of neural dynamics in the estimated dynamics matrix. The importance of flows in response to intrinsic (natural) or induced perturbations in capturing dynamies has been highlighted in Seung 1996). Yoon et al. (2013), Chaudhuri et al. [2019], G.aaal et al. [0023, as their evolution over time can uniquely identify dynamic structure.
Our results demonstrate that DSA effectively identifies when two neural networks have equivalent dynamics, whereas standard geometric methods fall short. DSA therefore has great relevance for neuroscience, providing another method to validate the fit of a model the neurophysiology data. additionally opens the door to novel data-driven analyses of time-varying neural computation. We expect our method to be interesting to computational neuroscientists due to its theoretical richness, valuable to deep learning researchers which to interpret and compare neural networks, and useful for experimental neuroscientists due to its simplicity of implementation and ease of interpretation.
theoretically backed and rigorously justified tool through
Contributions We develop a general method, DSA, for comparing two dynamical systems at the level of their dynamics. To do so, we introduce a modified form of Procrustes Analysis that accounts for how vector fields change under orthogonal transformations. We apply our method in four test cases, each demonstrating novel capabilities in neural data analysis: (a) Our method can identify dynamical similarities underlying systems with different geometries, which shape metrics cannot. (b) Conversely, DSA can distinguish systems with different dynamics despite similar geometries, which shape metrics cannot. (c) We show that our method empirically identifies topologiaal similarities and differences between dynamical systems, as demonstrated by invariance under geometric deformation and variance under topological transformation. (d) We demonstrate how to use DSA disentangle different learning rules in neural networks without supervision, by comparing how representations change across training.
2

---

2 Methods
2.1 Theoretical Background:
Koopman Operators and Dynamic Mode Decomposition
(DMD)
common approach to modeling nonlinear dynamics is to approximate them with linear systems. as those are more tractable and interpretable. One standard method involves finding fixed points of the dynamies, and then locally linearizing the system around each fixed point, determine whether infinitesimal changes diverge from or converge to the fixed point. Another is to describe nonlinear dynamies via locally linear models, separate one for each point state space. These methods do not generate description of the global dynamics. The former method also requires injecting perturbations, which may not be possible for experimental systems and pre-collected datasets.
The field of dynamical systems theory suggests different approach, on which a new class of data- driven methods are based. Specifically, Koopman operators theoretically embed nonlinear systems Hilbert space, which permits an exact and globally linear description of the dynamics (Koopman (1931)) ). Practical applications use the Dynamic Mode Decomposition (DMD) (Schmid [2110] to create a finite- diiite--imnsiinall embedding that approximates the Koopman operator (Brunton et al. (2016a 2017]. [nnder and Song (202T), Dubois et al. (2020). The DMD) identifies a linear transition operator for the dynamics: X (E At) AX(,, where X comprises a data matrix (consisting of functions of the observed states of the dynamical system). When applied to sufficiently rich X, the DMD can capture global dynamical structure. A key challenge of the method is constructing an appropriate space for X which is closed under the operation of A (a Koopman invariant subspace, [Brunton et al. (2016b]). We employ the Hankel Alternative View of Koopman (HAVOK, Brunton et al. 2017], Arbabi and Mezic (2017)) which constructs X as a function of the debey-emee of the observed variables (a Hankel Matrix), such that one column z(t - -r' where p is the number of included delays and 7 is the time-step between each measurement. The delay-embedding approach eliminates the difficulty of learning a nonlinear transformation of the data, and improves estimation capabilities over partially-observed systems (Kamb al. (2020). The latter capability arises from Takens' delay embedding theorem, which states that delay embedding is diffeomorphic to the original system- that is, a partially-observed system can be fully reconstructed using sufficient number of delays (Takens) [981]). However, our method could be used with range of embedding algorithms in place of delay embeddings.
into
of X = [-z(t) -z(t - r)
2.2 Dynamical Similarity Analysis (DSA)
Now, we introduce Dynamical Similarity Analysis, general method that leverages advances in both Statistical Shape Analysis and Dynamical Systems Theory to compare two neural systems at the level of their dynamics. Simply put, we use Koopman operator theory generate a globally linear embedding of nonlinear system, via delay embeddings as in HAVOK, then use DMD to define the linear transition operators in this space. Finally, we apply a novel extension of the Procrustes Analysis shape metric to the resultant DMDs from two systems. This set of steps constitutes DSA.
Suppose we have two dynamical systems defined by the following equations:
x = fxx,) X ERR y= g(y, YER"
(2)
We sample data from each system X E Rexkixtxxx and Y Rexkxxtxxm Here, e indicates the number of conditions observed, which in neuroscience and deep learning refer to different input stimuli or contexts. h;j indicates the number of trials per condition, and f; indicates the number of time steps observed per trial. Note that the observations must be sampled uniformly in time (Takens) 11)11) Next, we produce delay-embedded Hankel tensors with a lag of p: Ht, E Reec x(t-p- 1)xmp and likewise for Hy. Using the Hankel Alternative View of Koopman (HAVOK) approach (Brunton et al. (017]), we flatten all but the last dimension of H and fit reduced-rank regression models with rank the lige-timee-ela coordinates of the data, where the target the coordinates of the next time step (V'
V2 = AvV., where H. UVE.V VV. and Vx,r Vali, :
(3)
3

---

Only the rank of the HAVOK models explicitly need to be the same so that the two DMD matrices can be compared. Note that the eigen-ting time-delay coordinate formulation equivalent PCA whitening (Supplementary Information Section 12). which Williams et al. [202T] highlights to be important. The predictive capability of our model (via MSE, R², AIC, or others) can be used to identify the optimal rank and number of delays In our experiments, hyperparameters for the DMD were chosen to ensure both sufficient capability to fit the dynamics in question as well as tractable computation times. Fitting the HAVOK model to each system returns our DMD matrices: A., Ay E IR" which we compare using the modified Procrustes Analysis algorithm detailed next.
b. System mmmrrrson xt
System Represental Eiger-llim--Deen
NN
(507 en conddiing mamm
Dis Mas de
&1
C.
x
f(x)
x(x) = Kx(x)
Agec
c A.U
190 deay emtediing
DSA Ag)
min O20
|A4
CA,C $1,
d.
0.3
K3
Distance
Condition Averaged Hankel p DSA pw1 DSAp-
Figure 1: Schematic and example of DSA. a. Schematic of the DSA pipeline. b. Schematic of attractor reconstruction with delay embeddings. c. Schematic of the Koopman Operator linearization over time-delay coordinates. d,e. PCA trajectories (black lines) of a GRU (left) and Vanilla RNN (right) trained to produce sine waves of range of frequencies (Sussillo and Barak dots indicate center points of the limit cycles, colored by frequency of the output wave. f. Comparison of measured dissimilarity between the networks in d,e with (from left to right) Procrustes on the condition condition-averaged trajectories, Procrustes on the delay-embedded condition-averager trajectories, DSA on the original data, and DSA on the delay embedding. Lower is more similar. Each metric uses angular distance, which ranges from 0 to
2013 D. Colored
Procrustes Analysis over Vector Fields Procrustes Analysis is a shape metric which compares the geometric similarity of two datasets after identifying an optimal orthogonal transformation to align them (Dryden and Mardia (0166). To use it in DSA, we must modify the notion of orthogonal transformations to be compatible with linear systems. Recall that the Procrustes metric solves the following minimization problem (Williams et al. 2221
d(x, Y) = min CEO(n)
|X- - -CYl|e
(4)
Where O(n) is the orthogonal group (C = 1), F' indicates the Frobenius norm, and X, Y are two data matrices. A distance metric that is invariant to orthogonal transformations captures the intuitive notion that there no privileged basis across neural networks.
However, vector fields, represented by the DMD matrices Ag, A, above, transform differently than vectors X and Y (see Supplementary Information Section for an illustration). The invariance seek is that of a conjugacy, or change of basis, between linear systems, which reflected in the similarity transform, C. C-1 Note that this is special
of equation over linear systems. Thus we introduce a novel similarity metric, which we call Procrustes Analysis over Vector Fields:
d(Ag, Ag) = min CEO(M) A
(5)

---

The angular form of this distance can be written as:
As (CA,C-1) A |Ayl
d(A Ay) =
min arccos CEO(M)
(6)
where A.. (CA,,C-) is computed as CT, In the Supplementary Information Section 9 we prove an equivalence between DSA and the 2-Wasserstein distance between the two DMD matrices' cigenspectrum, in special cases. This connection highlights the grounding of DSA in
Koopman operator theory to measure conjugacy: when two dynamical systems are conjugate, their
Koopman eigenspectra are equivalent (Proof in Supplementary Information T0 Budišíć et al. [2012].
Lan and Mezić 2013)). We derive equation 5 from first principles and prove that it is indeed
proper metric in the Supplementary Information (o:: the proper metric property is crucial for various
downstream machine learning pipelines such as clustering or classification (Williams et al. [021]
Equations 5 and are nonconvex optimization problems similar to those in Jimenez et al. (013).
although we solve them with gradient-base optimization. Across our experiments, we found that the metric asymptotes in roughly 200 iterations using the Adam optimizer (Kingma and Ba (0114) with a learning rate of 0.01. To optimize over the orthogonal group, On). we utilize an extension of the Cayley Map, which parameterizes (O(n) (Supplementary Information 5.1) ..1. In our experiments, we compare the response of DSA with direct application of Procrustes to the data matrices it the most analogous shape metric. If only the identification of conjugacy is desired, metric properties can be relaxed by optimizing C' over the larger group of invertible matrices, GL(
3 Results
To demonstrate the viability of our method, we identified four illustrative cases that disentangle geometry and dynamics, and additionally performed an ablation analysis of DSA. First, a set of neural networks that are different by shape analyses but are dynamically very similar; second, a set of neural networks that are similar by shape analyses but are dynamically are very different; third, a ring attractor network whose geometry we can smoothly and substantially deform while preserving the attractor topology: finally, a line attractor network whose topology is transformed into a ring attractor by adding periodic boundary conditions.
3.1 Ablation Study
Importantly, both the reduced-rank regression and delay-embedding in DSA are required to isolate dynamical structure. We demonstrate this in Fig. T. where we compare the hidden state trajectories of an RNN and GRU solving the Sine Wave task (Sussillo and Barak (203)) using the standard Procrustes Analysis (via the netrepp packuge, |Williams et al. [2021). D. delay-embedded Procrustes Analysis (an ablation), DSA with no delay embedding (another ablation), and DSA score. To be exhaustive, we also computed a similarity metric that compares purely temporal information, defined as the angular Procrustes distance over the power spectrum of the RNN's 's hidden state trajectories. On the RNNs in Fig. [ this metric reported 1.41. Only DSA identifies the systems as highly similar, indicating that both the delay embedding and the DMD are necessary to abstract geometric particulars from each system.
3.2 DSA captures dynamical similarity between RNNs despite differences in geometry.
We trained a collection of 50-unit RNNs on the well-known 3-bit Flip Flop task from Sussillo and Barak [2013]. In this task, an RNN is presented with time series from three independent channels, each sporadically occurring pulses of either -1 or 1. The network's 's objective is to continuously output the last pulse value for each channel until it is updated by new pulse (hence the Flip-Flop name). Following Maheswaranathan et al. [2019]. We varied architecture (LSTM, GRU, UGRNN, Vanilla RNN), activation function (ReLU, Tanh), learning rate (.01,0.005), and multiple different seeds across each set of hyperparameters, ultimately collecting dataset of 240 networks. We only saved network if it solved the task with MSE less than 0.05. After training, tested all models on the same 64 input sequences and extracted the recurrent states, upon which we apply Procrustes and DSA in parallel to compare networks.
Our RNNs have different geometry but equivalent attractor topology (Maheswaranathan et al. (019)) ) 2h. displays the trajectories of individual trials in sample network in the first three principal
5

---

components, which capture on average 95.9% of the total variance. The computational solution to this task is characterized by eight stable fixed points at the vertices of a cube that correspond the eight possible unique memory states (29). Importantly, this cubic structure is preserved across all networks, even though particular geometric aspects of the trajectories may differ. Standard shape analyses differentiate architectures and activation function by geometry, which we replicated in Fig. 2b (architecture) and in the Supplementary Information (activation, 13
We fit individual HAVOK models with 75 delays and rank 100 to sampled trajectories from each network. For computational efficiency, we reduced the dimensionality of the network to 10 using Principal Component Analysis (PCA) before applying HAVOK. In Supplementary Information [4 we demonstrate that HAVOK's ' predictivity is robust to hyperparameter variation. Because the variance explained from the later PCs is negligible, they can be disregarded while preserving the original dynamics. This improves HAVOK's fit to the data and reduces the memory and time complexity of DSA. For each pair of networks, we compared these states directly using Procrustes Analysis and on the DMD matrices using Procrustes Analysis on Vector Fields. This yields two dissimilarity matrices (DMs), which we reduced to two dimensions using Multidimensional Scaling (MDS. Kruskal (1964)). We plot the results in this similarity space in Fig. 2 where each point is a trained network. As expected, the networks cluster by architecture (Fig. 2p) and activation function (Supplementary Information) when Procrustes is applied, indicating that aspects of their geometry differ. However, the networks intermingle under DSA (Fig. 2 inset zoomed in), and have pairwise dissimilarities close to zero, indicating that their underlying dynamics are equivalent, which is correct (Mabeswaranathan jet al. 2019]). We quantified these differences by applying linear SVM to the dissimilarity matrices and plotted the test accuracy in Fig. Fi.. This further underscores the differences between Procuutes and DSA in Fig. [2p and c, DSA's accuracy is close chance, whereas Procrustes achieves high accuracy on both labels.
a.
b.
Candition Averrage
C.
DSA
C.ya
Archtaciuure
Vanilanter
GRU
6-8-
STM
UGANN
2b
00
-0.2
00 Archiaadirre
Actindion
Same dynamies, different shape, only identified as similar with DSA. Dots indicate a single trained network. Analysis of RNNs trained on the 3-bit FlipFlop task. a. PCA trajectories of a single trained RNN. b. MDS projection of the dissimilarity matrix (DM) computed across andition-avveagee hidden states between each RNN (architecture indicated by color). In this view of the data, RNNs cluster by architecture. c. MDS embedding of the DM generated from DSA of the same set of RNNs in b. Here, RNNs do not cluster by architecture. d. Classification test accuracy of condition-ave -averaged Procrustes and DSA similarity spaces on both architecture and activation labels. Dotted lines indicate chance.
Figure 2:
3.3
DSA identifies dynamical differences despite geometric similarity
Next, we sought to display the converse, namely that our similarity method can identify differences in dynamical structure even when spatial analysis cannot. To do so, we examined a set of three noisy recurrent networks from Galgali et al. [2023]. These systems implement a bistable pair of attractors, line attractor, and single stable fixed point at the origin. These systems describe three strategies in a classical perceptual decision-making task from systems neuroscience: unstable evidence integration, stable integration, and leaky integration. Their dynamical equations are included in the Supplementary Information. Here, the inputs to the latter two networks were adversarially optimized so that the condition-averagee trajectories are indistinguishable from the first network (Fig. 3p.. These networks have intrinsic noise in their hidden states, which enables DSA to distinguish systems beyond condition averages. We simulated 200 noisy trajectories from 100 networks per class, each with randomly sampled parameters.

---

a.
Bistable
b.
Condition-Averaged
DSA
10
Nodel Type
0.75
attractor
point attractor
Line Attractor
Point Attractor
-3.03
-0.75
02
Figure 3: Different dynamics, same shape, only distinguished by DSA. a. Sample single-trial (faded) and anndiiion-average trajectories of the three systems in two dimensions (x and axes). Trials start at the center point and move away in direction depending on the condition (colored). b. Procrustes-MDDS on the dissimilarity matrices of the three network types. c. DSA-MDS of the three networks. d. Classification test accuracy of condition-averaged Procrustes and DSA similarity spaces. Dotted line indicates chance.
3.203-0 Gollo 0l000 2015 6506
E.0G CASS Che 000 100
1051
CSA
To confirm that the condition-aveeraeed trajectories are indistinguishable, we visualize the condition- averaged and single-trial trajectories in Fig. (3 for each system. We fit HAVOK models with 100 lags and rank of 50 for each network and computed DSA and Procrustes pairwise (5,000 total comparisons), As before, we plot a low-dimensional MDS projection of these dissimilarity matrices (3.. When the two types of similarities are compared side by side (spatial similarity, Fig. Bp. and DSA. Fig. BB). it is evident that only our method is capable of easily distinguishing dynamic types from the data alone. This is quantified by almost perfect test accuracy via linear classifier in Fig. B3. Thus, DSA can efficiently extract features that describe the underlying dynamical system from data alone, which better reflect the nature of the computation that is being performed in RNNs.
3.4 DSA is invariant under geometric deformations that preserve attractor topology
Next, we examined how DSA responds when smooth geometric deformations are applied to a dynamical system. Here, we chose to study a ring attractor network, a system known to track head direction in the brain, and whose topology is a ring embedded in neurons (Zhang) (1996). Chaudhuri et al. [219]. (Skaggs et al. 1995). The neural activity the ring is instantiated as bump of local activity, which is stable due to the localized and transiationall--symmeett profile. In each trial, we randomly sampled uniformly between 100 and 250, and drove the network with constant input and dynamical noise, leading the bump to rotate around the ring at a roughly constant velocity (we used code from Wang and Kang (2022). D. The network's ' dynamics are detailed in the Supplementary Information (16). Importantly, after simulation, the saved activations can be transformed by continuous deformation that preserves the ring topology while changing the geometry:
connectivity
(7)
1 + exp
The variable describes the synaptic activations of the ring network (the dynamical variable), and r describes the neural activations, which we compare with DSA and Procrustes. Here, 3 controls the width of the bump. As 3 increases, the width of the bump becomes progressively tighter, which makes the ring less planar (see Fig. 1 in Kriegeskorte and Weil [202T] for a depiction). We scaled the magnitude of 3 from 0.1 In the inset of Fig. 4b. we quantified the ring's 's dimensionality across B.
(E, A.)² Σ, X
measured as the participation ratio of the principal components' eigenvalues: PR
Gao
ct al. (0177).
After applying these transformations to simulated trajectories from our network, we applied DSA and Procrustes to respectively compare all Bs to the lowest ,3 in our dataset. In Fig. F we visualize the hidden state trajectory in the first two PCs as the ring is progressively transformed with 3. We fit HAVOK models with a lag of 10 and a rank of 1000 to trajectories at each level of deformation. When we compared the network using DSA and Procrustes, we found that only DSA is invariant across these transformations (Fig. 4p). While the value for DSA not strictly zero, this may due to approximation numerical error in HAVOK and the similarity metric, as it remains close to zero
7

---

b.0.
Procrustes
300
0.6
044
15
D5A
0.5
d.
1.2
1.0
Procrustes
0.4
0.2
DSA
0.0
0.2
0.4
0.6
0.8
1.0
Figure 4: DSA is invariant under geometric deformation of ring attractor, and sensitive to the transformation of a ring attractor into a line. Top row displays scaling of in the sigmoid, displays topological transformation of
making the ring progressively less planar. The bottom ring into a line via a. The network is a curve line attractor at a < 1, and is a ring attractor at a = a, Trajectories along the ring plotted in the top two PCs of the network, colored by magnitude of the deformation parameter. In e, trajectories are scaled for visualization purposes. b,d. Distance (D) to 3 0.1, and a 0.0 both Procrustes and DSA. Importantly, the topology only changes in the last row, at = 1.0. Inset b: Participation Ratio of the ring, natural continuous measure of dimensionality based explained variance ratios of PCA eigenvalues (Gao al. (2017)). Inset Persistent Homology Analysis of the ring: top 2 lifespans of Betti number features, indicating that the data becomes more tnn-like as a increases.
throughout (between 0.05 and 12). This suggests that DSA identifies similarity at the level of the topology of a dynamical system.
3.5 DSA responds to changes in topology
Finally, we assessed the converse to the previous analysis: does DSA respond to changes in topology? Using the same ring attractor network, we continuously varied the strength of its periodic boundary conditions to convert the ring into a curved line segment attractor by ablating some neurons from the network one end of the ring. Our ring is specified by cosine kernel that defines how each neuron connects its neighbors. This kernel has a length parameter! which specifies how far apart two neurons can connect. To completely eliminate the boundary conditions, / neurons must therefore be ablated. Using an ablation length as the number of neurons to ablate in the ring, we defined the following parameter to normalize the transformation across different network sizes.
a=1-i
(8)
When a 1, there is no ablation and the network is perfectly ring. When < 1, the network breaks into a line segment attractor. As a - the ring becomes progressively less curved. To ensure that network sizes were consistent across all values of a, we initialized the network size to be + before ablation, where n is the desired final number neurons. We simulated networks of sizes between 200 and 300 neurons. Instead of constant drive of input before, we drove the network with fixed magnitude input u whose sign flipped stochastically. We chose this input to prevent the bump of activity from getting trapped one end of the line attractors.
When we compared each value of a of the network to that DSA reports values close to 0 until becomes close to 1.0, when it jumps almost to x/2. We plotted these results Fig. 4 where panel depicts the low-dimensional visualization of the line and the ring, and d displays metric's response. Note that because the data are noisy, expected DSA
with DSA and Procrustes, we identified

---

to jump slightly before 1.0, which we empirically verified to be around a = 0.9 in our simulations. Here we used only 10 delays (for total embedding size between 2000 and 3000) and rank 100, but found that our results generalized to much larger DMD models as well. This further solidifies the notion that DSA captures topological similarity of two dynamical systems.
10
Aramder
66-
Groden DHOOR 1204 AAamoAur
50
Parometer 1
N0G1
Mepabi (2500) Sonsadoo (2512)
Figure 5: DSA unsupervisedly disentangles learning rules. a. Schematic example of different algorithms minimizing 3x + 2yy. b. DSA MDS plot of hidden activation observables from networks trained on ImageNet DSA MDS plot of the dynamics of learning in 3-layer linear networks. d. Test accuracy of a linear decoder on the DSA similarity space for the model classes. Bars indicate standard deviation over a wide range of DSA hyperparameters.
3.6 DSA can disentangle learning rules in an unsupervised fashion
It is still unclear what learning rules the brain uses. Recent progress includes the design of novel biologically-plasee
learning rules (Lillicrap et al. (2016), Hinton (2022), experiments and model- ing work that suggests signs of gradient-based learning in the brain (Sadtler et al. [2014), Humphreys let al. [2022). Payeur et al.) (2023), supervised methods to classify learning rules from aggregate statisties in neural data (Nayebi al. [2020]) and theory describing how representations evolve across training in different learning rules (Bordelon and Pehlevan [2022). Because artificial networks initialized with random seeds, their trajectories in the activity space across learning will almost always have different geometry, despite the fact that they may converge to the same minimum. However, because the trajectories arise from the same dynamical system, DSA should be able to capture similarities between learning trajectories from different initializations. We visualize a variety of learning rule trajectories in Fig. 5h to demonstrate how qualitatively different their dynamics can be. We applied DSA on two test cases from the literature to demonstrate that we can empirically dis- entangle artificial networks trained with different learning rules. To do so, we fit DSA to trajectories of the neural representations (or observables) across training epochs.
In the first case, we drew 21 aggregate statistics from the dataset from Nayebi et al. 20200] which describe the evolution of synaptic weights and activations of layers in various convolutional neural networks trained on multiple tasks with four learning rules. The learning rules utilized in the dataset include Adam (Kingma and Bal (2014]), Stochastic Gradient Descent with Momentum (Sutskever
ct al. (2013)) Information Alignment (Kunin et al.] [2020]), and Feedback Alignment (Lillicrap et al. (0166). ). For each analysis, we fixed one task and generated 50 datasets by randomly selecting (without replacement) subsets of trajectories for each learning rule. On each dataset, we fit a HAVOK model with rank of 45 and lag of 4 to trajectories of these statistics across training. We compared each model pairwise, which results in a dissimilarity matrix. After comparing trajectories of these observables across training with DSA, we visualize the MDS of the similarity space in Fig. 5p. The training dynamics clearly cluster by learning rule under DSA. Quantifying the degree of clustering, we could classify the learning rule with 88.5% test accuracy using only a linear classifier and 5-fold cross-validation. whereas the lincar SVM classifier in Nayebi et al. (2020) only reached ~ 55% accuracy (Fig. 2a. in their paper). The test accuracy over wide range of DSA hyperparameters was 81 % (Fig. 5)).
In the second case, we trained a set of small 3-layer feedforward neural networks (with hidden size of 100 units) trained solve multivariate regression task from Bordelon and Pehlevan [2022]. The learning rules in the second set include Gradient Descent, two forms of Feedback Alignment that vary based on the initialization of the random feedback weights (Boopathy and Fiete [2022 ), and
Hebbian learning (Hebb) 1949) ). For each learning rule, we trained 50 networks on 1000 epochs
with random initializations and randomly generated data. We fit HAVOK models with a delay of 100
and a rank of 500 onto individual networks' activations across learning. As above, we found that

---

the MDS visualization of these networks clustered by learning rule in the pairwise similarity space (Fig. (5)). The two types of Feedback Alignment cluster together, suggesting that their dynamics are similar, as might be expected. Once again utilizing linear classifier, we achieved 89% test accuracy after assigning both types of FA as the same class, and on average 80% aceuracy over a range of hyperparameters (Fig. (H... These results suggest the DSA could be used in hypothesis-driven fashion to assess learning rules in data recorded from biological circuits.
Discussion
We demonstrated that our novel metric, DSA, can be used to study dynamics in neural networks in numerous scenarios, including the comparison of dynamics between different RNNs, dynamics of single RNN across training, or dynamics of weight and activity changes feedforward networks over training. We performed an ablation study in Fig. D which demonstrates that both delay embedding and the DMD were required for DSA to succeed. While we applied our metric only to RNNs, we stress that it can be applied to any dynamical system, including diffusion models or generative transformers. Procrustes Analysis over Vector Fields could also applied to Koopman Embeddings identified via other algorithms, such as autoencoder (Lusch et al. (0188).
As suggested by Figs 2 4 and our theoretical grounding ( I0 DSA can capture similarity at the level of topology of the dynamics, also known as conjugacy. This can also be recognized from the methodology of DSA: the similarity transform conjugacy between two linear systems. In recent work, Redman et al. (026) used the 2- Wasserstein Distance over the Koopman eigenspectrum to compare learning dynamics of fully-connected networks trained to classify handwritten digits. This metric is quite analogous to ours in its theoretical motivation. In Supplementary Information [ we prove that their metric is a specific case of DSA. Thus, DSA quantifies how far two dynamical systems are from being conjugate.
As we demonstrate in Fig. our method could be used to quantitatively compare various hypotheses about the dynamics of a particular biological neural circuit (such as fixed point structure). by
calculating similarities of various RNNs or bespoke models to data (Pagan et al. 2022]. Schaeffer
et al. (0200). This task is typically computationally intensive, relying on assessing the dynamics of noise perturbations the neural state space (Chaudhuri et al.) [2019) Galgali
al. [2023) or
computing the locations of fixed points in task-optimized RNN model that has similar spatial
trajectories as the neural data (Mante et al. [20 [hhissnngmonkonn et al. (2017). Our method will
be useful in the analysis of biological neural systems, where only recorded neural activity is accessible. To our knowledge, DSA is the first that can be used to quantitatively assess different dynamical hypotheses. We suggest that this method should be used in conjunction with shape analyses when assessing fits of computational models to data. In future work, we plan to apply DSA to experimental data.
DSA could potentially be used for rapid alignment of Crapu-Co Interfaces (BCI) across days in subject. Non-stationary sampling of neurons from intracortical electrode shift, neuron death, and other noise plague longitudinal BCI trials, requiring the subject to relearn BCI control daily in a slow and tedious process (Biran et al. [Poge). et al. (2013). Two recent approaches (Willett et al. [2023), [Degenhart et al. (2020) utilized Procrustes Analysis their decoding and alignment pipeline. As demonstrated in Figs. 2 and 4 an alignment method that takes dynamics into account will be useful. Karpowicz et al. [2022] utilized LFADS to align dynamics across days, but DSA may provide equal performance with substantially higher data efficiency and less compute due to the method's simplicity.
In Section 3.6 we demonstrated how the DSA metric space could be used in hypothesis-driven fashion to compare neurophysiological data recorded across training to artificial neural networks trained with different learning rules. To do so, one would train variety of neural networks with different learning rules to solve the same task as the animal model. By applying DSA pairwise to each network and the neurophysiology data, and identifying which learning rules cluster to the experimental data, we can reject other learning rule hypotheses. DSA has multiple advantages over previous methods for this problem: Unsupervised feature generation can improve classification capabilities, and abstraction from geometrymakes the method more generically applicable across systems of different types and initial conditions.
10

---

Limitations It can be challenging to identify the optimal DMD hyperparameters, but it is not computationally intensive perform grid sweep. Additionally, we found that the ranks and delays required to achieve our results were relatively small. On V100 GPU, single DSA took between 30 seconds and few minutes, depending on the hyperparameters and size of the dataset.
Acknowledgments The authors grateful to Antonio Carlos Costa for bringing Koopman opera- tors to our attention, and to Sarthak Chandra, Raymond Wang, Aran Nayebi, Ho Kyung Sung, Will Redman, Tosif Ahamed, and members of the Fiete lab for helpful discussions and advice.
References
Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J. Majaj. Rishi Rajalingham, Elias Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy. Franziska Geiger, Kailyn Schmidt, Daniel L. K. Yamins, and James DiCarlo. Brain-score: Which artificial neural network for object recognition is most brain-like? bioRxiv, 2020. URLhttps: https::/ts/wwt http://wwwwboxxii.oo/o/o// biorxiv wwwooiii.org//oooo wwtg/conteen/eerly///000
01/02/407007
Rishidev Chaudhuri, Berk Gerçek, Biraj Pandey, Adrien Peyrache, and Ila Fiete. The intrinsic attractor manifold and population dynamics of canonical cognitive circuit across waking and sleep. Nature Neuroscience, 22(9):112215200 2019.
Alan D. Degenhart, William E. Bishop, Emily R. Oby, Elizabeth C. Tyler-Kabara, Steven M. Chase, Aaron Batista, and Byron M. Yu. Stabilization of brain-computer interface via the alignment of low-dimensional spaces of neural activity. Nature Biomedical Engineering, pages 672-685, 2020.
Marino Pagan, Vincent D Tang, Mikio C. Aoi, Jonathan W. Pillow, Valerio Mante, David Sus- sillo, and Carlos D. Brody. A new theoretical framework jointly explains behavioral and neural variability across subjects performing flexible decision making. bioRxiv, 2022. URL httts::/www.bioxtttt hwwpbbiwwwwoooooooooooooooooooet httpe//www.rreeottee//e////////e cow.eett/earlyy/222/////1//// 211/22222111//22222/22 11 28. 518207
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 2008.
Maithra Raghu, Justin Gilmer, Jason Yosinski. and Jascha Sohl-Dickstein. Sveca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Advances in neural information processing systems, 30, 2017.
Alex H Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. Generalized shape metrics on neural representations. In Advances Neural Information Processing Systems, volume 34, pages 4738-4750, 2021.
Lyndon Duong, Jingyang Zhou, Josue Nassar, Jules Berman, Jeroen Olieslagers, and Alex H Williams. Representational dissimilarity metric spaces for stochastic neural networks. arXiv preprint arXiv:2211.116633 2022.
Shun-Ichi Amari and Michael A Arbib. Competition and Cooperation in Neural Nets. Systems Neuroscience, Academic Press, J. Metzler (ed)., pages 119-165, 1977.
JJ Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sei. U. S. A., 81 (10):33088-3002 1984.
J J Hopfield and D W Tank. Computing with neural circuits: model. Science, 233(4764))625-633 1986.
H Seung. How the brain keeps the eyes still. Proc. Natl. Acad. Sci. U. A., 93322):13339-1133 1996.
Zhang. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: theory. J. Neurosci., 15:2112-2126, 1996.

---

Richard H. R. Hahnloser, Rahul Sarpeshkar, Misha A. Mahowald, Rodney J. Douglas, and H. Sebas- tian Seung. Digital selection and analogue amplification coexist cortex-inspired silicon circuit. Nature, 2000.
Yoram Burak and lla Fiete. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput. Biol., 502):e1000211, February 2009.
Xiao-Jing Wang. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 2002.
David Sussillo and I. F Abbott. Generating coherent patterns of activity from chaotic neural networks. Neuron, 6344):544-557, 2009.
Mark M Churchland, John Cunningham, Matthew T Kaufman, Justin D Foster, Paul Nuyujukian, Stephen Ryu, and Krishna V Shenoy. Neural population dynamics during reaching. Nature, 487 (7405):51-56, 2012.
Saurabh Vyas, Matthew Golub, David Sussillo, and Krishna V Shenoy. Computation through neural population dynamics. Annual Review of Neuroscience, 43:249-275, 2020.
Mikail Khona and lla R. Fiete. Attractor and integrator networks in the brain. Nature Reviews Neuroscience, 23(12):744-76 2022.
Aniruddh R. Galgali, Maneesh Sahani, and Valerio Mante. Residual dynamics resolves recurrent contributions to neural computation. Nature Neuroscience, 2023.
Igor Mezić. Spectral properties dynamical systems, model reduction and decompositions. Nonlin- ear Dynamics, 41:309-325, 2005.
Gregory Snyder and Zhuoyuan Song. Koopman operator theory for nonlinear dynamic modeling using dynamic mode decomposition. arXiv preprint 2r.1v:2110.08442 2021.
Steven L. Brunton, Marko Budišić, Eurika Kaiser, and J. Nathan Kutz. Modern Koopman Theory for Dynamical Systems, 2021. URL httt::t/taaxiii/// wwtp://rrg/abbbb//b/bbbe 12086
KiJung Yoon, Michael A. Buice, Caswell Barry, Robin Hayman, Neil Burgess, and Ila R. Fiete. Spe- cific evidence of oow-dimesiional continuous attractor dynamics in grid cells. Nature neuroscience, 16(8):1077-1084, 2013.
Bernard O Koopman. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences, 17(5):315-318, 1931.
Peter Schmid. Dynamic mode decomposition of numerical and experimental data, Journal of Fluid Mechanics, 656:5-28, 2010.
Bingni W. Brunton, Lise A. Johnson, Jeffrey G. Ojemann, and J. Nathan Kutz. Extracting spa- tial temporall coherent patterns in large-scale scale neural recordings using dynamic mode decomposition. Journal of Neuroscience Methods, pages 1-15, 2016a.
Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor, Eurika Kaiser, and Nathan Kutz. Chaos as an Intermittently Forced Linear System. Nature Communications, 8(1):19, 2017.
Pierre Dubois, Thomas Gomez, Laurent Planckaert, and Laurent Perret. Data-driven predictions of the Lorenz system. Physica D: Nonlinear Phenomena, 408:132495, 2020.
Steven L Brunton, Bingni W Brunton, Joshua L Proctor, and Nathan Kutz. Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PloS one, 11())x0150117, 2016b.
Hassan Arbabi and Igor Mezić. Ergodic Theory, Dynamic Mode Decomposition, and Computation of Spectral Properties of the Koopman Operator. SIAM Journal on Applied Dynamical Systems, 16 (4):2096-2126 January 2017.
12

---

Mason Kamb, Eurika Kaiser, Steven L. Brunton, and J. Nathan Kutz. Time Time-Delay Observables for Koopman: Theory and Applications. SIAM Journal on Applied Dynamical Systems, 19(2): 886-917, 2020.
Floris Takens. Detecting strange attractors in turbulence. In Dynamical Systems and Turbulence. Warwick, 1981.
David Sussillo and Omri Barak. Opening the Black Box: Low-Dimensional Dynamics in High- Dimensional Recurrent Neural Networks. Neural Computation, 25(3):626-649, 2013.
lan Dryden and Kanti Mardia. Statistical shape analysis, with applications R: Second edition. 2016.
Marko Budišié, Ryan Mohr, and Igor Mezić. Applied Koopmanism. Chaos: An Interdisciplinary Journal of Nonlinear Science, 22(4):047510, December 2012.
Yueheng Lan and Igor Mezić. Linearization in the large of nonlinear systems and koopman operator spectrum. Physica D: Nonlinear Phenomena, 2013.
Nicolas D. Jimenez, Bijan Afsari, and Rene Vidal. Fast Jacobi-type algorithm for computing distances between linear dynamical systems. In 2013 European Control Conference (ECC), pages 3682-3687, Zurich, 2013. IEEE.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6988. 2014.
Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Uni- versality and individuality in neural dynamics across large populations of recurrent networks. In Advances in Neural Information Processing Systems, volume 32, 2019.
J. B. Kruskal. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1):1-27, 1964.
W. E. Skaggs, J. Knierim, H. S. Kudrimoti, and L. McNaughton. A model of the neural basis of the rat's 's sense of direction. Advances in Neural Information Processing Systems, 7:173-180, 1995.
Raymond Wang and Louis Kang. Multiple bumps can enhance robustness noise in continuous attractor networks. PLOS Computational Biology, 18(10)11-38. 10 2022.
Nikolaus Kriegeskorte and Xue-Xin Wei. Neural tuning and representational geometry. Nature Reviews Neuroscience, 22(1 (1:1317773778, 2021.
Peiran Gao, Eric Trautmann, Byron Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. bioRxiv, 2017. URL httpps:/wwat biorxiv. wwtp:///oneeeeeeeer///////////1/
Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, 7(1): 13276, 2016.
Geoffrey Hinton. The forward-forward algorithm: Some preliminary investigations. arXiv preprint 133:4.2212.13345, 2022.
Patrick T. Sadtler, Kristin M. Quick, Matthew D. Golub, Steven M. Chase, Stephen I. Ryu, Elizabeth C. Tyler-Kabara, Byron M. Yu, and Aaron P. Batista. Neural constraints learning. Nature, 512 (7515):423-426, 2014.
Peter C. Humphreys, Kayvon Daie, Karel Svoboda, Matthew Botvinick, and Timothy P. Lillicrap. Bci learning phenomena can be explained by gradient-based optimization. bioRxiv, 2022. URL
https://www, http://wwwbbxxxxo/////////:t bwwrxiv. wwtp:xii..rgg/oooooooooeoo httg://oneent/aary///2/////////
12, 08, 519453
Alexandre Payeur, Amy L. Orsborn, and Guillaume Lajoie. Neural manifolds and gradient-based adap- tation neural-interface tasks. bioRxiv, 2023. URL http::t:ts:/s.iiiotottt http://wwwbbbbxi...oo http::/wwww..biirrivvoor//ottttt org/content/
early/2023/03// 000///22/2222 03. 532146
13

---

Aran Nayebi, Sanjana Srivastava, Surya Ganguli, and Daniel L Yamins. Identifying Learning Rules From Neural Network Observables. In Advances Neural Information Processing Systems, volume 33, pages 2639-2650, 2020.
Blake Bordelon and Cengiz Pehlevan. The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks, 2022. URL
httpaaaawivvrrrar/aaaaaaaa .22177
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance initialization and momentum in deep learning. In Proceedings of the 30th International Conference on Machine Learning, pages 1139-1147, PMLR, 2013.
Daniel Kumin, Aran Nayebi, Javier Sagastuy- BBrena, Surya Ganguli, Jonathan Bloom, and Daniel Yamins. Two routes to scalable credit assignment without weight symmetry. In International Conference on Machine Learning, pages 5511-552) PMLR, 2020.
Akhilan Boopathy and lla Fiete. How to Train Your Wide Neural Network Without Backprop: An Imput-Weight Alignment Perspective. In Proceedings of the 39th International Conference on Machine Learning. PMLR, 2022.
Donald O. Hebb. The organization of behavior: A neuropsychological theory. Wiley, New York, 1949.
Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9(), 2018.
William T. Redman, Juan M. Bello-Rivas, Maria Fonoberova, Ryan Mohr, Ioannis G. Kevrekidis, and Igor Mezić. On equivalent optimization of machine learning methods, 2023.
Rylan Schaeffer, Mikail Khona, Leenoy Meshulam, Brain Laboratory International, and Ila Fiete. Reverse-engineeringg recurrent neural network solutions to a hierarchical inference task for mice. In Advances in Neural Information Processing Systems, 2020.
Valerio Mante, David Sussillo. Krishna V. Shenoy, and William T. Newsome. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 2013.
Warasinee Chaisangmongkon, Sruthi K. Swaminathan, David J. Freedman, and Xiao-Jing Wang. Computing by robust transience: How the fronto-parietal network performs sequential, category- based decisions. Neuron, 2017.
Roy Biran, David C. Martin, and Patrick A. Tresco, Neuronal cell loss accompanies the brain tissue response to chronically implanted silicon microelectrode arrays. Experimental Neurology, 2005.
János A. Perge, Mark L. Homer, Wasim Q. Malik, Sydney Cash, Emad Eskandar, Gerhard Friehs, John P. Donoghue, and Leigh R. Hochberg. Intra-day signal instabilities affect decoding perfor- mance in an intracortical neural interface system. Journal of Neural Engineering, 2013.
Francis R. Willett, Erin M. Kunz, Chaofei Fan, Donald Avansino, Guy H. Wilson, Eun Young Choi, Foram Kamdar, Leigh R. Hochberg, Shaul Druckmann, Krishna V. Shenoy, and Jaimie M. Henderson. A high-performance speech neuroprosthesis. preprint, Neuroscience, 2023. URL http://biorxiv org/ http://ooooooooooo///o/o/ doi/:////oooo/..ooo 11110.1111111111111111 01. 0224999.2499
Brianna M. Karpowicz, Yahia H. Ali, Lahiru N. Wimalasena, Andrew R. Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma, Lee E. Miller, and Chethan Pandarinath. Stabilizing brain-computer interfaces through alignment of latent dynamics. bioRxiv, 2022. URL https
7/www. biorxiv. org/ http:///ntnnneeerrr/////////////
04. 06. 48738888
14


Reviews:
[{"review_id": "5DZUAIKKoD", "reviewer": "Reviewer_f1wF", "summary": "The authors propose Dynamical Similarity Analysis (DSA), a method for assessing the dynamical similarity between dynamical systems. The method combines ideas from the data-driven Koopman operator literature and a Procrustes-type distance between linear operators. By focusing on dynamics rather than geometry, the method is able to reveal the similarities between RNNs of different architectures trained to perform the same tasks and distinguish between dynamical systems with similar time-averaged states but different underlying dynamics.", "questions": "1. Figure 1 uses angular distance -- where is this defined?\n2. There are several comments made about noise in the manuscript. For example, l58: \"DSA factors noise into its estimate of neural dynamics.\" Also l172. I remain confused about how noise interacts with different aspects of the model. For example, how does Takens' theorem interact with noise?\n3. How are lags and rank chosen for each experiment. Are the results robust to these hyperparameters? They seem to vary widely from experiment to experiment.\n\nSuggestions/comments:\n* Define what shape metrics are, or perhaps explicitly define one common one.\n* Condition is not defined in the setup.\n* Quantifying dissimilarity: Figure 3: MDS is a good visualization, but there should also be predictions as in Figure 2 or distances as in Figure 1. This should be uniform across figures.", "limitations": "* Paragraph starting at l333: I would expect one would need to fit a model that fits task-relevant dynamics. That is, DSA would probably require some modifications before it is applied to neural data.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method is simple and appealing. Appealing use of time delay embeddings -- transform the temporal task into a geometric task.\n2. DSA works very well on most of the experiments: Figures 1, 2, 3, and 5.", "weaknesses": "1. Should compare to baseline spectral method, for example PCs of CPSD matrix...\n2. Figure 4: DSA is not zero. l262 speculates about numerical approximation error, but this seems to be fairly large. Ideally, the line for DSA would stay very close to zero. If this is not due to numerical approximation error, should this be taken as a failure case of DSA? The text mentions the low variability of the DSA distance across values of beta and C, but it seems like the magnitude is more relevant in this example.\n3. The manuscript would improve with better theoretical grounding. For example, something in the vein of what is proposed in the paragraph starting at line 317 would greatly improve the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful feedback and suggestions. We are glad that the simplicity of the method was appealing, and we hope that our responses to each of your comments will allow you to further appreciate DSA.\n\nWe appreciate the suggestion to add a spectral method of comparison, and agree that it would be valuable to compare to additional baseline metrics. After some investigation, though, we have to admit we are unclear on what the process would be for the baseline method you suggested. Would it be possible for you to elaborate on your suggestion by pointing us to a specific reference? This would help us incorporate your feedback better.\n\n> Figure 4: DSA is not zero.\n\nWe agree that this is a problem, and we are happy to share that we have solved it since submitting the paper. We will update the figure accordingly--now, DSA reports \\~0.06 with a standard deviation of 0.01 across the deformation values (Fig 4) . The error was in fact numerical approximation error due to optimization, and we solved it by including a multilayer perceptron in our pipeline, which makes the loss landscape more amenable to gradient descent. In our open source code (to-be-released accompanying the paper), we implemented a unit test asserting that a matrix $A$ and its similarity transform $CAC^{-1}$ would be identified as similar up to a threshold, which now passes. \n\n> The manuscript would improve with better theoretical grounding.\n\nWe agree with the reviewer on this weakness and are happy to share that we have identified theoretical grounding. The proof is quite straightforward and we can describe it in words here, but we will certainly include the formal proof in the paper: For two topologically conjugate systems $f(x),g(y)$ related by a diffeomorphism $y = \\phi(x)$ , their Koopman operators $K_x,K_y$ have the \\textit{exact same} eigenvalues, and the eigenfunctions are related by the same diffeomorphism $\\phi$ (Proposition 7, Budisic et al. 2012, \"Applied Koopmanism\"). This implies that the DMDs - the finite approximation of the Koopman operator - are related via the similarity transform, as similar matrices have identical eigenvalues.  Thus, we can identify topological conjugacy of two nonlinear dynamical systems with DSA.\n\n> Figure 1 uses angular distances -- where is this defined? \n\nWe only defined this in the appendix, section 3 (eq. 15). We agree that this may be confusing and are happy to add it into the methods section.\n\n> comments made about noise in the manuscript\n\nWe agree that our description here was confusing, and are happy to clarify both here and in the paper. \nNoise (in particular, small noise perturbations) is important for estimating the DMD appropriately by reducing degeneracy in the dynamics matrix estimation. As a motivating example, consider estimating the dynamics of a system observed close to a fixed point. Without noise, the regression would be degenerate as the system is barely changing. But with small noise perturbations, there is enough variance in the observed data to enable the construction of an appropriate dynamics matrix.  A more complex argument for the importance of noise is also described by Galgali et al., Nat. Neuro (2023), from which we identified the examples in Fig. 3.  Essentially, without small noise perturbations, our method would not be able to distinguish the examples in Fig 3, as the condition-averaged dynamics are exactly equivalent.\n\nThe reviewer also brings up an interesting question about how noise interacts of theoretical backing for our method, such as Takens' theorem. In the specific case of Taken's theorem, it seems as though there have been attempts to rigorously characterize the quality of the state space reconstruction at varying signal to noise ratios (see Casdagli et. al. 1991). However, again noting that our description was confusing, we wish to clarify that we meant to claim not that our method is robust to large amounts of noise, but rather that the method is able to harness small noise perturbations in order to obtain appropriate models of the dynamics.\n\n\n> How are lags and rank chosen for experiment\n\nLike most DMD papers, we chose them heuristically, based on our knowledge of the dimensionality of the system (both in terms of intrinsic manifold and number of neurons), and based on how well the DMD model predicted the next step data (3 metrics here, MSE, R\\^2 and Correlation). However, the results are robust to hyperparameters--in Appendix Fig section 11 we showed that the predictivity metrics are optimized across a wide range of ranks and lags, which are reasonable proxies for how well DSA will do. We are aware of algorithms that optimize these hyperparameters based on predictivity metrics (Ahamed et al., Nature Physics 2020) but chose not to use them here, as picking hyperparameters by hand was surprisingly easy to get our method to work.\n\n> Suggestions\n\nRegarding your suggestions, we agree with all of them. We defined the Procrustes Analysis (a particular shape metric, William et al., 2021) in equation 3 but realize that we did not explain its function very well, so we will elaborate upon it in our revisions. Specifically, we plan to add an intuitive explanation of Procrustes describing that it captures distance in a manner that is invariant to orthogonal transformations, which is relevant because the coordinates of the space are not important for the setting of comparing neural representations. We also agree that we need to define what we mean by condition, and will do this in the revision--a condition in systems neuroscience experiments is the context that the experimenter varies (input stimuli and desired output mapping, e.g. the variable input drive in Fig 3--one condition has input $u$ with positive magnitude, the other condition has negative input). Finally, we agree that we should quantify the discriminability achieved by DSA in a panel in Fig. 3 and will add that into the paper. "}, {"review_id": "4a1iOcSYEI", "reviewer": "Reviewer_cz18", "summary": "The authors developed a new computational tool named Dynamical Similarity Analysis (DSA) to measure the similarity between two systems focusing on the dynamics. They constructed the method by combining and modifying Dynamical Mode Decomposition and Statistical Shape Analysis. Their method successfully identified dynamical similarities between systems with different geometries, and conversely distinguished systems with different dynamics with similar geometries. They also used their method to distinguish learning rules underlying measured neural dynamics.", "questions": "None. Very nice work.", "limitations": "The authors adequately addressed limitations.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "I enjoyed this paper, which I found to be both elegant and useful. This work is a novel combination of previously developed techniques. This work is significant because the standard approach for comparing recurrent neural networks merely focuses on the spatial similarities of latent states, ignoring the importance of the difference between temporal similarities. Their proposal avoids the limitations of traditional tools by accounting for neural dynamics. Overall, the authors have clearly delivered their statement, proposed method, experiment, and results. The method is well-grounded and the ideas behind the proposed technique are well-explained and justified. The writing was clear, and the figures were largely appealing (although some of the labels became small and/or blurry). They analyzed when their method is superior to traditional methods, and used simple tasks to demonstrate this superiority. The distinguishing of learning rules from data was particularly nice.", "weaknesses": "The authors could better explain and motivate the conventional spatial Procrustes Analysis, as that would help some readers take the next step for the temporal version. They could clarify the interpretation of their similarity transform CAC^-1, as I felt that the explanation that C^-1 “rotates [the vectors’] positions as well” was not a great explanation.\n\nMinor: It’s strange to talk favorably in Figure 2 about DSA giving better accuracy but having lower accuracy scores, when discriminating between identical computations. If pairs of networks are correctly identified as “the same dynamics” then NOT distinguishing the networks should have higher accuracy. You might want “discriminability” on the vertical axis of Figure 2d.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and are very happy that they found the paper elegant, significant, and useful! We found their suggestions for improvements quite helpful as well, and hope that we can implement them in a manner that they agree with: \n\nIn their point that some of the figure's labels became small and blurry, we think they are referring to Figure 1? If so, we uploaded a figure that was too large and have since compressed it--we believe that will solve this problem. We are also happy to increase label sizes as needed. \n\nWe also agree that we did not explain and motivate Procrustes well, instead skipping over it to motivate our new metric. In the main paper, we can add an intuitive explanation of Procrustes describing that it captures distance in a manner that is invariant to orthogonal transformations, which is relevant because the coordinates of the space are not important for the setting of comparing neural representations. \n\nRegarding the interpretation of $CAC^{-1}$ , we believe that the quoted interpretation of the similarity transform goes very well with our Figure 1 in the Appendix, and can move that explanation there so that they align. We can replace the interpretation in the main paper with a more mathematically motivated one based on homomeorphisms  between conjugate dynamical systems, in this case linear systems, or another alternative interpretation that is clearer.\n\nFinally, we agree that our description of \"good\" and \"bad\" in Figure 2 does contradict standard perceptions with respect to classification accuracy, and would appreciate your feedback on the rewording. We suggest that we could change the phrase \"the networks intermingle\" in L231 to \"the networks are indiscriminable\", and then reference Fig 2d in that line. We can chance the vertical axis to \"discriminability\" as you suggested, and add one more sentence at the end of section 1 explaining that it is desirable to our metric to have discriminability close to chance in this example. Alternatively, we could define a metric \"indiscriminability\" as 1-test\\_acc, but this may be more confusing? "}, {"review_id": "D0Jni5xG3j", "reviewer": "Reviewer_mRC1", "summary": "Understanding how different neural populations process a particular computation is critical for the study of brain computation, the development of brain-inspired technologies like brain-computer interfaces (BCI), and AI applications. Existing methods compare the underlying representations based on the spatial geometry of the latents. However, they fail to capture the dynamics, which are believed to drive the specific computation. To overcome this limitation, this work introduces a new analysis method that first extracts the dynamical structure of the neural network and then aligns these representations for comparison. They test the model when applied to several case studies training RNNs across different tasks and architectures. The proposed analysis can capture similarities between the computations across different networks while other models fail.", "questions": "The case studies using RNNs show the strengths of the proposed analysis in comparison to other methods. However, it is not always the case that the new analysis is superior to the alternatives. It would be interesting to explore cases when it isn't and why that is the case. As described, the modified Procrustes alignment seems to only support pairwise comparisons. If that is the case, are there limitations when comparing multiple networks? Does one have to set a reference network to compare all the others?", "limitations": "The authors thoroughly address the limitations on hyperparameter training and computational cost. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly presented and technically sound. The method was tested and shown to work well when applied to artificial neural networks trained to solve different tasks and architectures, resulting in different dynamics and geometries. The analysis expands on two existing methods and combines them to understand the comparison between neural dynamics, which support the computation. Understanding similarities and differences between neural networks solving similar tasks is critical for the fields of neuroscience and AI. In this work, they showed how this analysis can identify overlap between computation solutions when other alternative methods fail. Moreover, the authors suggest that it could be used to understand biological network dynamics.", "weaknesses": "While the authors provide compelling results supporting the relevance of this metric when applied to RNNs, they have not applied it to neural data. Since this project is largely motivated by the potential study of biological neural circuits and the application to BCIs, applying the analysis to biological data is critical to fully grasp the scope of applications of the proposed method and therefore its significance. As mentioned in the manuscript, the analysis is \"[...] useful for experimental neuroscientists due to its simplicity of implementation and ease of interpretation due to its linearity.\" However, while its linearity provides the aforementioned benefits, it also imposes some limitations that could be interesting to explore. Lastly, when it comes to the application of alignment for BCI applications, it is important to note that the reported training time is between one hour and one day, making it impossible to use in its current state for practical applications when daily alignment is presumably needed. Along the same line of thought, it would be interesting to explore and note the data demands for the new method to work robustly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and greatly appreciate that they think the paper is clear and technically sound. We hope that our response will clarify some of the weaknesses and questions that the reviewer mentioned:\n\n> Applications to neural data\n\nWhile we haven't applied to neural data, we should note that the first example in the disentangling learning rules section (Fig 5a) utilized data from large-scale CNNs, trained on Imagenet (for example, a deep resnet, Nayebi et al., 2020). Although even this isn't quite as complex as biological neural data, we hope that this setting points to DSA's viability on neural data as well. We also agree that the application to neural data is one of the most exciting and important applications of our method, and are currently developing follow up work involving neural data.\n\n> limitations of the linear method\n\nWhile the model itself is linear, we note that the DSA gains nonlinear power via the delay embedding, which acts similar to a kernel function in an SVM. However, it is still worth thinking about DSA's limitations, and we're actively working on a deeper theoretical analysis. \n\n> BCI limitations and training time\n\nWe apologize for the lack of clarity in our reported training time--we should note that the time to fit *one* DSA comparison is on the order of a minute or less, provided the number of optimization iterations is not absurdly large. We noted in the limitation that the total time per experiment (i.e., a figure) was between one hour and one day because we were performing tens of thousands of comparisons. For example, in Figure 2, we compared 240 neural networks pairwise, or around 29 thousand comparisons in total. Naturally, this takes a long time to compute, but each individual comparison is relatively fast, and we have GPU capability which drastically improves performance. We can clarify this in the limitations, and we hope that this explains what we meant by one hour to one day of computation.\n\n> data demands\n\nWe agree that the question of how our method scales is a very interesting one to consider. We believe that the key limiting factor is in the DMD, which should scale with respect to the underlying dimensionality of the data manifold. We are actively studying this in a followup project--thank you for your suggestion.\n\n> superiority of DSA vs alternatives\n\nWe agree that DSA is not always superior. We wish to emphasize that we do not intend to replace shape metrics, but rather to supplement them with a new comparison method to be done in parallel. Geometry is of course an important aspect to consider, and there exist many use cases for geometric methods. The goal of our paper was to highlight that if one cares about dynamics, there are situations in which geometric methods will not suffice to capture relevant similarities or dissimilarities at the level of the underlying dynamics. \n\n> Pairwise comparisons in DSA\n\nThis is a similar concern for standard shape metrics--by virtue of being distance metrics, they can only compare pairwise. When comparing multiple networks (which we did in a few settings) there are a couple clear options, as you've mentioned: (1) compare pairwise, which results in a representational dissimilarity matrix of $R^{n \\times n}$ (Fig 2,3,5) or (2) we can fix a reference network (Fig 4) for a representational dissimilarity matrix of $R^{n \\times 1}$. In our open source code (which we are releasing), we've made it easy to do either one, with the option of setting hyperparameters for each model individually or in aggregate. "}, {"review_id": "HtgS1NXBoF", "reviewer": "Reviewer_TLG4", "summary": "In this paper the authors propose a similarity metric to compare dynamical systems – in particular neural networks. They use a set of recent methods to transform a general dynamical system into a basis in which the dynamics is approximately linear. They compare two dynamical systems by comparing their linear dynamics in this transformed space by seeing how well one system can be turned into the other using only an orthogonal transformation. They test their metric on a variety of settings.", "questions": "-\tIt seems like the scale of the MDS dimensions is being argued to be meaningful in your plots. Beyond some vague intuition that bigger means more variance, are the dimensions meaningful? What do their scales correspond to?\n-\tHow well do the linearised dynamics capture the underlying network dynamics? In my limited knowledge of the HAVOK methodology (gathered from Steve Brunton’s excellent youtube videos) the system cannot usually be modelled as a linear system without including a forcing term. Do you find the same thing? If so do you not think it could be that key aspects of the dynamics are entering through the dynamics of the forcing term, slightly weakening the power of your method. Regardless, I think it would be good to persuade people that these approximate dynamics you are comparing are actually good approximations.\n-\tI did not buy one claim, could you elaborate:\n“By linearity of PCA, the dynamics of the original system within the PC subspace are preserved” - Line 224.\nTake the limit, say the dynamics of the system is just circling round an ellipse and you perform PCA to reduce that to 1D, the major axis of the ellipse. Sure, the dynamics on that 1D are the same as the original system but surely you lose something in this projection? And wouldn't that potentially be true of your original system too?\n-\tYou claim that all networks that solve the 3-bit Flip-Flop task have the same 8 fixed point structure, line 219, do you have any evidence for this? Or is this from Maheswaranathan et al. 2019? If from Maheswaranathan, make it clear, cite them straight after you make the claim!\n-\tWhy did you not choose figure 3 to also back up your claim that DSA might be capturing similarity at a topological level? (line 317)\n-\tRelated to the topological claim, my intuition must be off. Humour me for a moment and ignore the time lagging: if you had activity rotating on some ring, then morphed the behaviour to become ellipsoidal, the topology would be the same, but the rotation matrices would be different. Further, the two matrices could not be mapped into one another by an orthogonal transformation. As such, in my intuition your method would (potentially very reasonably) say these were different dynamics even if the topology was the same. Given this, why do you think your method is going to care only about topology? Is it the time-lagging doing some magic?\n\n", "limitations": "The authors include a section of limitations that sound reasonable.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "I think the idea is a good one.\nI think it has been well executed.\nThe experiments were well chosen.\nThe writing was generally pretty good.", "weaknesses": "I don’t understand why section 2.3 was separate from section 3. It feels like section 2.3 played two roles – advertising the set of experiments that were going to test this new metric in interesting ways, and describing the experimental setup. The first of these is done in the first paragraph, and the second should, in my opinion, be moved into section 3. The separation between setup and results meant I had to keep flipping back, and you repeated yourself for no reason. It would make much more sense to me if each experiment was introduced and its results were discussed together.\n\nI thought some of the experimental details were also unclear, and generally you rely too much on the reader recalling in detail many of the papers you cite. In particular:\n1. I did not know what the 3-bit flip flop task was before reading this, and had to go find another paper that describes it. It seems crazy not to put a short description in the paper, even if it is in the appendix if page limits are the issue.\n2. When trying to understand the ring attractor I eventually decided that r and s in equation 5 were phi(g) and g in equation 25. But then why change notation? And if I was right could this be made clear? If not, it was obviously not clear enough for this dumbo to understand what was going on. \n3. Further, in appendix 9 you say “in our deformation analysis we only changed phi after simulation, which does not affect the ring topology but modifies the geometry”\nI had no idea what this meant. In my head simulation is running the network dynamics, so you’re saying you only changed the non-linearity after you’d simulated, in which case it would have… zero effect. I must be misunderstanding?\n\nThe claim that DSA measures topological structure of the dynamics would be fairly remarkable and does seem to be backed up by the experiments. However it doesn’t seem at all proven – as you yourself point out in the discussion. In fact, what exactly is being compared through the dynamics of the eigenspace of these lagged timeseries is a little opaque to me – and I might wager to you too. Given this lack of certainty, I have two specific places I think the writing could use a fudge word, like in the discussion where you say “DSA _may_ be capturing similarity at the level of topology”. Namely:\n1. Line 251: We demonstrate DSA compares two dynamical systems at the level of their topology.\n2. Line 266: This suggests that DSA identifies similarity at the level of the topology…\n\nCould you just write '_may identify similarity_', or '_is consistent with comparing at the level of topology_', because the pure claim “We demonstrate DSA compares two dynamical systems at the level of their topology” does not seem well founded.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are thankful for the reviewer's extensive feedback and appreciate their comments--we believe we can answer all of their questions, which correspond to valuable improvements to the paper.\n\n> I don't understand why section 2.3 was separate from section 3...\n\nThis is a reasonable point and we agree that we can restructure. Our intention was to follow the style of previous papers that leave technical descriptions of experiments in Methods alone. However, we agree that it would be more readable (and less redundant) to combine them. Thank you for the suggestion!\n\n> I did not know what the 3-bit flip flop task was before reading this...\n\nWe greatly appreciate this feedback - it is important to ensure the reader is clear on the task structure so that our results can be fully understood. We will introduce the task in more detail, and can add another panel (similar to Fig. 1a in Maheswaranathan et al., 2019) describing the task visually.\n\n> When trying to understand the ring attractor I eventually decided that r and s in equation 5 were phi(g) and g in equation 25...\n\nThis is correct and we apologize for the oversight. Thank you for catching the error.\n\n> Further, in appendix 9 you say \"in our deformation analysis we only changed phi after simulation, which does not affect the ring topology but modifies the geometry”...\n\nWe apologize for the lack of clarity and will  reword this. In our experiment, we applied a new nonlinearity to the synaptic activations to get a different output of the rates of the neurons. So we transformed the data on which we applied DSA, while still using the same underlying trials--you can also think of this as a modification of how we observed the data. The goal was to demonstrate invariance to smooth deformations. In Fig. 4b, the Procrustes distance increases with the magnitude of the nonlinearity, which verifies that the data is deforming . We hope this makes sense and are happy to clarify further.\n\n> The claim that DSA measures topological structure of the dynamics would be fairly remarkable... but is not proven...\n\nWe thank the reviewer for this important point and have since identified a proof that we will include. We can also add fudge words as you suggested. The proof is simple and we will include the formal proof in the paper: For topologically conjugate systems related by a diffeomorphism, their Koopman operators have the *exact same* eigenvalues, and the eigenfunctions are related by the diffeomorphism (Proposition 7, Budisic et al. 2012). This implies that the DMDs--the finite approximation of the Koopman operator--are related via the similarity transform.\n\n> It seems like the scale of the MDS dimensions is argued to be meaningful... what does it correspond to?\n\nAs you identified, the dimensions are linear projections of the data, so a small scale implies small dissimilarity between systems. Thus in Fig 3b, being clustered around 0 means that the RDM is made up of mostly zeros. If they were scattered farther apart it would suggest that the models could be very different from one another. In revisions, we can display some statistics over the RDM or explain this logic in more detail. \n\n> How well do the linearised dynamics capture the underlying network dynamics?...\n\nIn Appendix 8, we swept a range of DMD hyperparameters and found that the DMD fit the flip-flop system relatively well. Nevertheless, you are correct that most nonlinear systems are not linearizable and the DMD cannot perfectly fit. *However*, based on our analysis, that is not a problem for DSA, as the relevant outcome is for the system to converge to an approximation of the Koopman operator. Thus even if the dynamics are not perfectly fit, the similarity transform distance will identify conjugate systems as similar.\n\nWe should clarify that forcing is used to generate *autonomous* nonlinear dynamics, in that the system can be fed its own predictions and thus reproduce the dynamics. Time-delay observables have been used to approximate of the Koopman operator without forcing in several settings (Arbabi & Mezić 2016, Brunton et. al. 2016) and thus the omission of the forcing term should not (and indeed does not) impact DSA.\n\n> I did not buy one claim, could you elaborate:...\n\nWe thank the reviewer for this concern and very much agree. Here's what we meant: in a high-dimensional system that has low dimensional dynamics (for example, 99% of the variance is explained by the first 5 dimensions), we can effectively project to that 5-d subspace and the system will evolve in the same way as in the ambient space. This is because the dynamics components orthogonal to those dimensions is negligible. But in general this is not true, as in your example. We can delete this line if it would eliminate confusion, or can add a section in the appendix about the viability of PCA in DSA in certain conditions (manifold hypothesis).\n\n> You claim that all networks that solve the 3-bit flip-flop task have the same 8 fixed point structure, line 219, do you have any evidence for that?\n\nYes, this was shown in Maheswaranthan et al., 2019 and Sussillo and Barak 2013--they perform an extensive numerical analysis. We apologize for the oversight and thank you for the suggestion -  we will cite directly after making this claim.\n\n> Why did you not choose figure 3 to also back up your claim...\n\nWe apologize for this--it was a semantic distinction ('similarity' versus dissimilarity). Nevertheless, we should connect the concept of capturing topological similarity to Fig. 3.\n\n> Related to the topological claim, my intuition must be off...\n\nThis is an interesting scenario, and is related to Fig. 4a. In DSA, we compare the two systems on the eigen-time-delay coordinates. In doing so, singular values are factored out, which \"re-circularizes\" your ellipse. In Williams et al. (2021), they solve a similar problem by whitening the data, and fitting HAVOK in the eigen-time-delay coordinates is equivalent to PCA whitening on the data."}]

Author rebuttal:

Meta-review: The authors introduce Dynamical Similarity Analysis (DSA), a novel method for the dynamics of neural networks and other dynamical systems. By combining advanced techniques like the data-driven Koopman operator and Procrustes Analysis, DSA effectively captures dynamical similarities in various architectures and can distinguish different underlying dynamics, offering a robust tool for understanding neural computations and testing recurrent neural network models. This is important work that will be of broad interest to NeurIPS audience and beyond.
Decision: Accept (poster)