Title: GeoDynamics: A Geometric State‑Space Neural Network for Understanding Brain Dynamics on Riemannian Manifolds
Venue: NeurIPS 2025
Accepted: yes
Keywords: Geometric Neural Network

State Space Model

Brain Dynamics Analysis

Riemannian Manifold
Abstract: State‑space models (SSMs) have become a cornerstone for unraveling brain dynamics, capturing how latent neural states evolve over time and give rise to observed signals. By combining deep learning’s flexibility with SSMs’ principled dynamical structure, recent studies have achieved powerful fits to functional neuroimaging data. However, most approaches still view the brain as a set of loosely connected regions or impose oversimplified network priors, falling short of a truly holistic, self‐organized dynamical system perspective. Brain functional connectivity (FC) at each time point naturally forms a symmetric positive definite (SPD) matrix, which lives on a curved Riemannian manifold rather than in Euclidean space. Capturing the trajectories of these SPD matrices is key to understanding how coordinated networks support cognition and behavior. To this end, we introduce *GeoDynamics*, a geometric state space neural network that tracks latent brain state trajectories directly on the high‑dimensional SPD manifold. *GeoDynamics* embeds each connectivity matrix into a manifold‑aware recurrent framework, learning smooth, geometry‑respecting transitions that reveal task‐driven state changes and early markers of Alzheimer’s, Parkinson’s, and autism. Beyond neuroscience, we validate *GeoDynamics* on human action recognition benchmarks (UTKinect, Florence, HDM05), demonstrating its scalability and robustness in modeling complex spatiotemporal dynamics across diverse domains.

GeoDynamics : A Geometric State-Space Neural Network for Understanding Brain Dynamics on Riemannian Manifolds

Abstract

State-space models (SSMs) have become a cornerstone for unraveling brain dynamics, capturing how latent neural states evolve over time and give rise to observed signals. By combining deep learning's flexibility with SSMs' principled dynamical structure, recent studies have achieved powerful fits to functional neuroimaging data. However, most approaches still view the brain as a set of loosely connected regions or impose oversimplified network priors, falling short of a truly holistic, self-organized dynamical system perspective. Brain functional connectivity (FC) at each time point naturally forms a symmetric positive definite (SPD) matrix, which lives on a curved Riemannian manifold rather than in Euclidean space. Capturing the trajectories of these SPD matrices is key to understanding how coordinated networks support cognition and behavior. To this end, we introduce GeoDynamics , a geometric state space neural network that tracks latent brain state trajectories directly on the high-dimensional SPD manifold. GeoDynamics embeds each connectivity matrix into a manifold-aware recurrent framework, learning smooth, geometry-respecting transitions that reveal task-driven state changes and early markers of Alzheimer's, Parkinson's, and autism. Beyond neuroscience, we validate GeoDynamics on human action recognition benchmarks (UTKinect, Florence, HDM05), demonstrating its scalability and robustness in modeling complex spatiotemporal dynamics across diverse domains.

1 Introduction

The human brain is a complex, dynamic system with distinct structural regions specialized for specific functions, which are locally segregated yet interconnected to process diverse information [36]. Over recent decades, understanding the brain's functional mechanisms has been a central focus in neuroscience. Functional magnetic resonance imaging (fMRI), a widely used non-invasive technique, measures blood oxygen level-dependent (BOLD) signals over time, which are linked to neural activity. While initial research emphasized BOLD signals, the focus has shifted toward functional connectivity (FC), which captures co-activations across brain regions [3]. Studies using Pearson's correlation to quantify FC have shown that functional brain networks are not static but exhibit dynamic topology changes, even in task-free states [4]. These dynamic changes have been associated with brain disorders, offering insights into underlying neurobiological processes [8].

Efforts to model brain dynamics have focused on two main approaches: (1) analyzing temporal fluctuations in BOLD signals and (2) capturing topology changes in evolving FC matrices. While

BOLD signals track neural activity, they often struggle with noise and intrinsic fluctuations. Neural mass models, for instance, describe brain dynamics using non-linear equations but often ignore spatial dependencies [63]. FC matrices, on the other hand, reveal network-level interactions and, when extended to dynamic FC (dFC), track connectivity evolution over time using methods like sliding windows [42]. Recent advances, such as a geometric-attention neural network proposed in [15], relate FC topology changes to brain activity. However, sliding window techniques remain sensitive to window size, which can hinder the detection of subtle brain state changes.

The widespread success of recurrent neural networks (RNNs, Fig. 1 (a)) [59], including long shortterm memory (LSTM) [31] and gated recurrent units (GRU) [14], in sequential modeling tasks such as natural language processing (NLP), has inspired numerous efforts to apply these architectures for characterizing brain dynamics [47; 48]. Recently, state space models (SSMs) (as shown in Fig.1 (b, black solid box)) [27; 26] have emerged as a powerful tool for capturing a system's behavior using hidden variables, or 'states", marked as s t (i.e., s ( t ) ), which effectively model temporal dependencies in sequential data with well-established theoretical properties. These models have gained significant attraction in fields like computer vision (CV) and natural language processing (NLP) due to their ability to represent complex temporal patterns. A more inclusive literature survey can be found in the Appendix A.1.

Relevant work of SSM on brain functional studies. While SSMs have achieved success in CV and NLP applications, their use in brain functional studies has primarily focused on eventrelated fMRI data [22; 35], limiting their applicability to resting-state fMRI (rsfMRI). To address this, [64] combined an auto-encoder for learning BOLD signal relationships with a hidden Markov model (HMM) for state transitions, but the separate training of these components reduced efficiency and neglected brain network spatial structures. Similarly, [66] proposed a linear SSM using variational Bayesian methods to infer effective connectivities from EEG and fMRI data. However, this approach struggles to capture the complex dynamics between evolving functional connectivity and underlying cognitive or behavioral outcomes.

𝑥

!

𝑠

!

𝑠

"

𝑠

#

𝑠

$

…

MLP

Downstream

tasks

(a)

𝑥

"

𝑥

#

𝑥

!

𝑠

!

𝑠

"

𝑠

#

𝑠

$

…

Downstream

tasks

(b)

𝑥

"

𝑥

#

𝑠

%

𝑥

$

𝑠

%

𝑦

!

𝑦

"

𝑦

#

𝑥

$

𝑦

$

𝐴

𝐶

𝐵

𝐷

𝐴

𝐴

𝐴

𝐵

𝐵

𝐵

𝐶

𝐶

𝐷

𝐷

𝑋

!

𝑆

%

𝑌

!

𝑆

$

𝑋

$

𝑌

$

…

Euclidean space

Manifold space

…

~

~

~

~

~

~

~

~

Our work. The dynamic nature of complex system cannot be understood by thinking of the system as comprised of independent elements. Rather, an approach is needed to utilize knowledge about the complex interactions within a system to understand the behavior of the system overall. In light of this, modeling the fluctuation of functional connectivities on the Riemannian manifold provides a holistic view of understanding how brain function emerges in cognition and behavior. In this paper, we integrate the power of geometric deep learning on Riemannian manifold and the mathematical insight of SSM to uncover the interplay between evolving brain states and observed neural activities. First , our method is structural in that we propose to learn intrinsic FC feature representations on the Riemannian manifold of symmetric positive-definite (SPD) matrices [17], which allows us to take the whole-brain wiring patterns into account by considering each FC matrix as a manifold instance. Second , our method is behavioral in that we leverage the SSM to model temporal dynamics. As shown in Fig. 1 (b), SSMs operate through two core ODEs, the state equation and the observation equation, which describe the relationship between the input x ( t ) (short for x t ) of the dynamic system and the system output y ( t ) (short for y t ) at a given time t , mediated by a latent state s ( t ) (short for s t ). Taken together, our contributions have threefold. (1) We present a novel geometric deep model by integrating state space model and manifold learning. By incorporating Riemannian geometry, our deep model provides an in-depth insight into system dynamics and state transitions, enhancing the model's ability to capture both temporal and spatial complexities in a data-driven manner. (2) We replace

the Euclidean algebra of conventional SSMs with Riemannian geometric algebra (accompanied by theoretical analysis) to effectively capture the spatio-temporal information, which allows us to better handle irregular data structures and harness the geometric properties of SPD matrices. (3) We have significantly improved the computational efficiency compared to manifold-based deep models by using modern machine techniques such as geometric deep model (Sec. 3.1) and SPD-preserving attention mechanism (Sec. 3.2).

We apply our proposed method ( GeoDynamics ) to two types of system dynamics: brain dynamics and action recognition [6; 28]. While brain dynamics is our primary focus, action recognition serves as a validation task to assess the method's generalization performance across different domains. In the application of understanding brain dynamics, upon which we refer to as GeoDynamics , we have evaluated model performance on the large-scale human brain connectome (HBC) databases, including one Human Connectome Project [79] and four disease-related resting-state fMRI data: (1) Alzheimer's Disease Neuroimaging Initiative (ADNI) [52], (2) Open Access Series of Imaging Studies (OASIS) [46], (3) Parkinson's Progression Markers Initiative (PPMI) [50], and (4) the Autism Brain Imaging Data Exchange (ABIDE). For action recognition, we use three classic human action recognition (HAR) datasets including the Florence 3D Actions dataset [61], the HDM05 database [53], and the UTKinect-Action3D (UTK) dataset [75]. Our GeoDynamics has achieved significant results across both brain dynamics and action recognition tasks, demonstrating its effectiveness and practicality. These applications on both neuroscience and computer vision highlight the scalability and robustness of GeoDynamics in understanding complex spatio-temporal dynamics across diverse systems.

2 Preliminary

2.1 Classical State Space Model

The continuous-time state space model is defined as

where s ( t ) ∈ R N is the system state, x ( t ) ∈ R is the control input, and y ( t ) ∈ R M the output (we follow the single-input/single-output convention unless otherwise stated). The matrices A ∈ R N × N , B ∈ R N × 1 , C ∈ R M × N , and D ∈ R M × 1 (often D =0 ) encode transition and observation, as sketched in Fig. 1(b). While Eq. (1) assumes Euclidean variables, many neuroscientific signals, notably functional connectivity matrices, are naturally manifold-valued (SPD), motivating a geometry-aware extension.

2.2 Riemannian Geometry on the SPD Manifold

Let M = SPD( N ) denote the manifold of N × N SPD matrices. We write X,Y ∈ M and use the Stein metric [11] for computational efficiency:

This distance avoids repeated eigendecompositions while preserving key geometric structure.

Group action and isometric 'translation'. Let G = GL( N ) be the general linear group acting on M via g.X := gXg ⊤ . Under the Stein metric, such actions are isometric, restricting to the orthogonal subgroup O ( N ) ⊂ G preserves distances, d ( g.X, g.Y ) = d ( X,Y ) for all X,Y ∈ M (proof in Appendix A.2). We call this an isometric translation on M and denote it by

Weighted Fréchet mean (wFM). Given { X n } N n =1 ⊂ M and nonnegative weights { w n } N n =1 with ∑ n w n = 1 , the wFM is

Assuming { X n } lies in a suitable geodesic ball, existence/uniqueness holds (see Ref. [1]).

SPD convolution on M . For matrix-valued features, the SPD convolution at layer k is

with kernel H ∈ R θ × θ . To preserve SPD structure, we parameterize

so the learned parameter is Z while H is guaranteed SPD, consequently X ( k ) remains SPD (proof in Appendix A.3).

3 Method

Geometric formulation of state space dynamics. We extend the classical state space formulation to the Riemannian manifold of SPD matrices. In this setting, the system input X ( t ) , system state S ( t ) , and system output Y ( t ) are no longer vectors in R N , but elements of the SPD manifold M = SPD( N ) , equipped with the Stein metric. This design enables the system dynamics to evolve in a geometrically consistent manner that respects the intrinsic structure of functional connectivity representations. Formally, at time t , we denote: X ( t ) ∈ M , S ( t ) ∈ M , Y ( t ) ∈ M . The temporal evolution of the system is driven by a set of learnable operators defined on M , replacing the Euclidean linear operators of conventional SSMs.

3.1 State Update and Observation on the SPD Manifold

Let τ denote the length of the temporal receptive field (i.e., sliding window). We model the evolution of S ( k ) and Y ( k ) at discrete time step k ∈ { 1 , K } through two components: (1) Temporal aggregation of past states and inputs using the weighted Fréchet mean (wFM) to ensure intrinsic averaging along geodesics rather than Euclidean linear combinations. (2) Group action translation on the manifold to represent state transitions and input effects in an isometry-preserving manner.

The state update and observation equations are defined as:

where F denotes the wFM operator on M (Eq. 4), and T denotes the translation on the manifold induced by orthogonal group actions (Eq. 3). The learnable parameters { A j , B j , C j , D j } control how the history of states and inputs contributes to the current state.

Weighted Fréchet mean aggregation. Given a set of SPD matrices { Z j } k j = k -τ and nonnegative weights { w j } k j = k -τ with ∑ j w j = 1 , the weighted Fréchet mean is defined as:

where d ( · , · ) is the Stein geodesic distance on M . Unlike Euclidean averaging, this operation produces an intrinsic barycenter on the manifold, ensuring that the aggregated representation remains SPD.

Group action translation. To propagate states forward in time, we introduce a translation operator T :

where g ( V ) is an orthogonal transformation inferred from V . Since g ( V ) ∈ O ( N ) , this operation is an isometry under the Stein metric and guarantees that the output remains SPD. The translation acts as a multiplicative update on the manifold, analogous to additive transitions in Euclidean SSM.

Discretized dynamics and control. We parameterize the temporal dynamics by discretizing the continuous-time operator using a matrix exponential scheme. Specifically, for each step ∆ :

This formulation ensures stable temporal integration while preserving geometric structure, allowing smooth evolution of the system's hidden states on M . The role of ˜ A is to propagate the previous hidden state, while ˜ B injects control-dependent information from the input. Their influence is integrated within the wFM in Eq. (7) to produce a manifold-consistent transition.

Task readout via logarithmic mapping. The output Y ( K ) is an SPD matrix, which cannot be directly passed to a standard classifier. We therefore map it to the tangent space at the identity using the logarithmic map:

where Y ( K ) = ΦΛΦ ⊤ is the eigendecomposition. This yields a symmetric matrix in Euclidean space that preserves local manifold geometry.

For classification task and brain state decoding, we employ a softmax layer:

where vec( · ) denotes vectorization of the symmetric matrix and Q is the number of task classes or clinical outcomes. Model training uses the cross-entropy loss:

where o iq is the ground truth label. E denotes the number of subjects/samples.

Global convolutional reformulation. Although the above formulation describes the step-wise state evolution, it can be equivalently expressed as a convolution operation over time [25]. Expanding the recurrence gives:

To ensure SPD structure, we constrain each kernel K r,l to be SPD by parameterizing

where ϵ > 0 is a stability constant. The nonlinearity is introduced via the exponential map exp( · ) on the Riemannian algebra, which guarantees that the output remains SPD after each convolutional block [15]. Frobenius normalization is applied to control eigenvalue growth and maintain numerical stability during training.

3.2 SPD-Preserving Attention

To enhance the model's ability to identify informative disease-related spatio-temporal patterns, we introduce a SPD-preserving attention (SPA) module embedded within the manifold convolutional backbone. Unlike conventional attention mechanisms that are defined in Euclidean space, SPA operates directly on the SPD manifold and is designed to preserve the positive-definite structure of the signal throughout the attention process. Given the convolutional response X ∗K on M , we define the attention weights as

where diag( ρ, X ) pads the borders of the SPD matrix with zeros of size ρ -1 and introduces a small positive diagonal component to ensure strict positive definiteness (see Appendix A.5). The exponential operation exp( · ) plays a role analogous to the sigmoid function, smoothly mapping the weighted response into the interval [0 , 1] while retaining manifold consistency. The resulting δ ( · )

acts as a soft mask that adaptively modulates local connectivity patterns. The attention-weighted representation is then computed as

where ⊙ denotes elementwise multiplication. Because δ ( · ) is strictly positive and bounded, and the base convolution is SPD-constrained, ˜ X remains SPD after modulation. This preserves both the local geometry (SPD structure at each time point) and global temporal consistency.

Disease-relevant pattern localization. Many neurological and psychiatric disorders, including neurodegenerative diseases such as Alzheimer's, are characterized by localized but propagating disruptions in functional connectivity . These disruptions often manifest as: (1) spatially specific abnormal hubs or subnetworks (e.g., default mode, limbic regions), (2) temporal shifts in network activation or coupling, (3) altered covariance structure in FC trajectories reflecting pathological propagation. In conventional SSM or GNN models, such disease-related signals may be smoothed out by global averaging or fixed-weight message passing. In contrast, the SPA mechanism learns spatially and temporally adaptive weighting functions directly on the SPD manifold, enabling: Enhanced sensitivity to local network deviations : the multiplicative modulation amplifies abnormal signal components (e.g., hyper- or hypo-connectivity in key subnetworks) while attenuating background fluctuations. Preservation of intrinsic geometry : since both the attention and the convolution are SPDpreserving, pathological patterns expressed as topological changes (e.g., altered covariance structure) remain valid on the manifold, avoiding distortions induced by Euclidean projections. Temporal adaptivity : δ ( · ) is computed at each time step, enabling the network to detect evolving abnormal FC trajectories associated with disease progression. Importantly, the learned attention weights δ ( · ) are anatomically and temporally interpretable, i.e., high attention scores highlight regions or subnetworks where deviations from healthy connectivity trajectories are most pronounced. This makes SPA not only an effective inductive bias for improving predictive performance, but also a tool for identifying candidate disease biomarkers.

3.3 Summary of the GeoDynamics Framework

The proposed GeoDynamics integrates: (1) Geometric state evolution: Input, system states, and outputs are represented as SPD matrices evolving on M , ensuring geometric consistency. (2) Manifold-aware operators: Temporal aggregation via weighted Fréchet means and state transitions via orthogonal group actions guarantee that system trajectories remain on the manifold. (3) Stable discretization: Matrix exponential schemes provide stable integration and controllable temporal dynamics. (4) Efficient convolutional formulation: Aglobal kernel expansion enables parallel training and scalable implementation. (5) Geometric attention: SPA enhances interpretability and sensitivity to task-relevant regions.

This formulation provides a principled and computationally efficient way to model non-Euclidean temporal dynamics, making it particularly well suited for functional connectivity sequences and other manifold-valued time series.

4 Experiments

4.1 Experimental Setup

Dataset. We apply our method to two types of datasets including human brain connectome (HBC) and human action recognition (HAR), more detailed data information is shown in Table 4 of Appendix A.6. For HBC dataset. We select one dataset of healthy young adults and four disease-related human brain datasets for evaluation: the HCP Working Memory (HCP-WM) [79], ADNI [52], OASIS [46], PPMI [50], and ABIDE [20]. We selected a total of 1,081 subjects from the HCP-WM dataset. The working memory task included eight task events. Brain activity was parcellated into 360 regions based on the multi-modal parcellation from [24]. For the OASIS (924 subjects) and ABIDE (1,025 subjects) datasets, which are binary-class datasets, one class represents a disease group and the other represents healthy controls. In the ADNI dataset, subjects are categorized based on clinical outcomes into four distinct cognitive status groups. The PPMI dataset also consists of four classes. We employ Automated Anatomical Labeling (AAL) atlas [67] (116 brain regions) on ADNI, PPMI, ABIDE datasets, while Destrieux atlas [18] (160 brain regions) are used in OASIS to verify the scalability

of the models. For HAR dataset. We validate the performance of the proposed GeoDynamics on three widely-used HAR benchmarks: the Florence 3D Actions dataset [61], the HDM05 database [53], and the UTKinect-Action3D (UTK) dataset [75]. The Florence 3D Actions dataset consists of 9 activities performed by 10 subjects, with each activity repeated 2 to 3 times, resulting in a total of 215 samples. The actions are captured by the motion of 15 skeletal joints. For the HDM05 dataset, we follow the protocol from [71], focusing on 14 action classes. This dataset contains 686 samples, each represented by 31 skeletal joints. Lastly, the UTKinect-Action3D dataset comprises 10 action classes. Each action was performed twice by 10 subjects, yielding a total of 199 samples.

SPD matrices construction . For HBC dataset . Each fMRI scan has been processed into N mean time courses of BOLD signals, each with T time points (where N represents the number of brain parcellations), we employ a sliding window technique to capture functional brain dynamics. Specifically, we construct a N × N correlation matrix at each time point t ( t = 1 , . . . , T ) based on the BOLD signal within the sliding window, centered at time t . This results in a sequence of FC matrices encoding the functional dynamics for each scan, represented as X = { X ( t ) | t = 1 , . . . , T } ∈ R T × N × N , in Fig. 2 (a). For HAR dataset. HAR datasets exhibit variability due to differences in action duration, complexity, the number of action classes, and the technology used for data capture. Therefore, we first apply a preprocessing step following [56] to obtain the SPD matrices. This step involves fixing the root joint at the hip center (red dashed circle in Fig. 2 (b)) and calculating the relative 3D positional differences for all other N -1 joints. For each timestamp t = 1 , . . . , T , we obtain a 3 × ( N -1) -dimensional column vector representing the relative displacements of the joints. Then, we compute covariance matrices using the method proposed in [56] to yield the SPD matrices. After that, we apply a sliding window technique to capture the dynamics over time, resulting in a sequence of SPD matrices X = { X ( t ) | t = 1 , . . . , T } ∈ R T × (3( N -1)) × (3( N -1)) , as illustrated in Fig. 2 (b).

Time

…

SPD Matrices

…

𝒳 = {𝑋 𝑡

∣ 𝑡 = 1, … , 𝑇

}

(b)

i

j

Manifold space

𝑺(𝟎)

BOLD signals

Spontaneous brain activity

i

j

Time

FC Matrices

…

𝒳 = {𝑋 𝑡

∣ 𝑡 = 1, … , 𝑇}

fMRI

(a)

(c)

𝑺(𝟓)

𝑺(𝟏𝟓)

𝑺(𝑻)

Learning the system dynamics

M

Comparison methods and evaluation metrics. For HAR dataset. There are some popular methods for HAR, such as multi-part bag-of-pose (MBP) [61], Lie group [69], shape analysis on manifold (SAM) [19], elastic function coding (EFC) [2], multi-instance multitask learning (MML) [78], Tensor Representation (TR) [45], LieNet[34], SPGK [72], ST-NBNN [73], GR-GCN [23] and DMT-Net and F-DMT-Net [80]. We also include Bi-long short-term memory (Bi-LSTM) [5] and part-aware LSTM (P-LSTM) [62]. For HBC dataset. We stratify the comparison methods for HBC into two groups: spatial and sequential models. Spatial models focus on capturing brain dynamics. Traditional GNNs like GCN [44] and GIN [77] are included for their ability to handle structured data. Subgraph-based GNNs like Moment-GNN [41] focus on identifying local patterns, while expressive GNNs like GSN [7] and GNN-AK [81] enhance subgraph encoding for better expressivity. SPDNet [16], a manifold-based model, is chosen for managing high-dimensional data. Plus, an MLP serves as a simple, generic baseline. Sequential models target temporal dynamics in BOLD signals. 1D-CNN captures temporal patterns, while RNN [59] and LSTM [31] handle sequential dependencies. MLPMixer [65] integrates both temporal and spatial information, and Transformer (TF) [68] captures global dependencies through attention. Mamba [25], vanilla SSM, is included for its ability to model system dynamics over time. Two dynamic-FC methods, STAGIN [43], NeuroGraph [60]. Three brain network analysis methods BrainGNN [49], BNT [40], and ContrastPool [76]. More details are shown in Appendix A.7.

Evaluation metrics. For Florence and UTKinect datasets, we adopt the standard leave-one-actor-out validation protocol as outlined in [23]. This method generates Q classification accuracy values, which are averaged to produce the final accuracy score. For HDM05 dataset, we follow the experimental setup from [32], conducting 10 random evaluations. In each evaluation, half of the samples from each class are randomly selected for training, with the remaining half used for testing. In all HBC

experiments, we utilize a 10-fold cross-validation scheme, reporting accuracy (Acc), precision (Pre), and F1 score to provide a thorough evaluation of model performance across various datasets.

4.2 Results on Human Brain Connectivity Datasets

We investigate brain dynamics across both healthy and disease-related cohorts using task-based and resting-state fMRI data.

Task-based fMRI. We first evaluate the task-based working-memory dataset (HCP-WM) using sixteen representative methods. The results in Fig. 3 (green colors) show that sequential models achieve markedly higher accuracy than spatial models (pair-wise t -test, p < 10 -4 ), with performance gains up to 30%. Our proposed GeoDynamics achieves the best overall performance.

Interpretation. From a machine learning perspective, sequential models may better capture the temporal dependencies in BOLD signals, which are inherently dynamic during task execution. In contrast, spatial models rely on static functional connectivity patterns that are less sensitive to rapid task-induced variations. Biologically, task-based fMRI paradigms elicit specific neural responses related to cognitive processes such as attention and memory, leading to pronounced fluctuations in brain activity that align naturally with sequential modeling.

Precision

F1-score

HCP-WM

ADNI

OASIS

PPMI

ABIDE

98.26

98.18

98.16

97.77

97.80

97.77

81.20

83.18

78.71

80.40

82.37

78.46

89.60

87.38

87.34

89.26

89.33

86.47

71.35

76.07

70.60

73.31

70.63

68.63

71.04

72.29

70.97

70.95

70.33

70.02

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.4

0.5

0.6

0.7

0.8

0.9

1.0

** *** ***** ***** *****

**** ***** ***** ***** ***** * **** *

**** ** **

*

** ***** **

*

* *** ****** **** ******

**** **** * **** ****** *

*** *** **

***** ***

*

*** **

** *

*

*

**

**

*

* *** ****** **** ******

**** **** * **** ****** *

*** *** **

***** ***

** **

** *

*

*

*

**

*

*

*

𝒑 < 𝟎. 𝟎𝟓

*

1D-CNN

RNN

LSTM

Mixer

1

Mamba

GCN

GIN

GSN

MGNN

GNN-AK

SPDNet

MLP

STAGINNeuroGraph GeoDynamics

Neurodegenerative diseases. Next, we focus on early diagnosis of neurodegenerative diseases (ND), including Alzheimer's Disease (AD) (yellow and purple colors) and Parkinson's Disease (PD) (pink colors), using resting-state fMRI. We assess the classification between cognitively normal (CN) and neurodegenerative (ND) groups. Spatial models show slightly better average performance than sequential models ( p = 0 . 37 ), although the difference is not statistically significant. Notably, our GeoDynamics substantially outperforms all baselines, achieving a significant improvement over the second-best method ( p < 0 . 05 ).

Interpretation. Unlike task-based fMRI, resting-state fMRI measures spontaneous neural activity, reflecting intrinsic connectivity rather than stimulus-driven dynamics. This may explain why spatial models perform competitively in ND classification tasks. From a biological standpoint, neurodegeneration primarily manifests as gradual network disconnection rather than transient dysfunction [55; 12]. Functional disruption often precedes measurable cognitive decline by many years [70]. The results in Fig. 3 thus highlight the clinical promise of deep learning models for detecting early ND-related alterations in large-scale brain networks.

Neuropsychiatric disorders. We further analyze Autism Spectrum Disorder (ASD) using the ABIDE dataset. As shown in Fig. 3 (blue color), spatial models slightly outperform sequential ones, with SPDNet consistently ranking second only to our GeoDynamics . Both SPDNet and GeoDynamics share two key design principles: (1) manifold-based representation learning that preserves the intrinsic geometry of functional connectivity matrices (Fig. 2 (c)), and (2) spatio-temporal modeling that

captures the evolution of connectivity patterns over time. The consistent advantage of these two approaches suggests that robust spatio-temporal representation, grounded in Riemannian geometry, is crucial for reliable diagnosis of neuropsychiatric conditions.

HCP-WM

OASIS

PPMI

ABIDE

ADNI

Default mode

Sensorimotor

Cerebellum

Somatosensory

Central executive

Frontal lobe

Temporal lobe

Visual

Interpretation. Neuropsychiatric disorders such as Autism and Schizophrenia are characterized by atypical neural connectivity and abnormal variability in BOLD dynamics [58; 51; 38]. These alterations affect both spatial topology and temporal synchronization, suggesting that effective diagnostic models must jointly capture both aspects. In contrast, neurodegenerative diseases primarily disrupt large-scale connectivity due to neuronal loss, making static spatial features more informative. Integrating such disease-specific pathophysiological insights is therefore essential for model design and interpretation.

Interpretable attention patterns. Finally, we visualize the attention weights δ ( · ) from the SPA module (Sec. 3.2) to identify critical brain regions and connections across datasets. The top-20 strongest connections are mapped onto the brain (Fig. 4). In HCP-WM, the dominant connections reside within the default mode network (DMN, blue) and central executive network (orange), both essential for working-memory tasks. In AD (OASIS and ADNI), the most affected regions include the DMNand somatosensory cortex (green), suggesting that AD may influence sensory processing and bodily awareness in addition to memory loss. In PD, key connections appear in the sensorimotor area (red), frontal lobe (purple), DMN, and cerebellum (black), indicating that PD involves both motor and cognitive-emotional dysfunctions. In Autism, prominent connections emerge in the temporal (brown) and visual cortices (yellow), consistent with known deficits in language, perception, and social interaction.

Interpretation. Despite disease heterogeneity, certain shared patterns emerge across neurodegenerative conditions, particularly AD and PD. The proposed attention mechanism not only highlights disease-specific alterations but also uncovers common pathways underlying different disorders. Such interpretability could provide valuable insights into shared pathophysiological mechanisms and guide hypothesis-driven neuroscience research.

4.3 Ablation Studies

Sliding window size (PPMI dataset) : We examined the effect of varying the sliding window size on model performance. As shown in Table 1, GeoDynamics demonstrates relative robustness to window size, achieving optimal performance with moderate values (typically 35-45). This stability likely arises from the SSM module's ability to capture dynamic temporal characteristics, reducing sensitivity to specific window lengths.

Multi-class classification (ADNI dataset) : We extended the binary classification task to multiple clinical stages. As reported in Table 2, GeoDynamics consistently achieves higher accuracy, precision,

and F1-score across all classes compared with both spatial- and sequence-based baselines. This demonstrates robust generalization to more challenging multi-class settings.

Model complexity : We systematically varied the total number of trainable parameters by adjusting hidden-state dimensionality in Mamba. In the final column in Table 3, we compare GeoDynamics against the best-tuned Mamba configurations at the same parameter budget. This demonstrates that our geometry-aware state-space formulation yields measurable gains in predictive performance, even when network capacity is held constant.

4.4 Model Validation

To evaluate the generality of our framework, we applied it to the HAR dataset using standard joint-coordinate benchmarks (Fig. 5). Despite the differences from neuroimaging data, GeoDynamics remains competitive, owing to its unified treatment of spatio-temporal dynamics and manifold geometry. By embedding 3D joint positions on a Riemannian manifold and employing a global convolution kernel, the model captures coordinated movements across distant joints while effectively suppressing noise. This demonstrates the broad applicability of our geometric state-space approach beyond neuroimaging. Furthermore, our method captures higher-order correlations among 3D joint coordi-

MBP

SAM

EFC

MML

TR

GeoDynamics

EFC

ST-NBNN

SPGK

Bi-LSTM

GR-GCN

LieNet

SPDNet

P-LSTM

DMT-Net

F-DMT-Net

Florence

UTKinect

HDM05

1.0

0.8

---0.6

0.4

nates over time and models spatio-temporal co-occurrences through the tailored convolution kernel, enhancing robustness to noisy joints and improving overall action recognition accuracy.

More experimental details are provided in Appendix A.7-A.8.

5 Conclusion

This work presents a geometric deep model of SSM, GeoDynamics , for understanding behavior/cognition through deciphering brain dynamics. In line with theoretical analysis, our method integrates the principles of geometric deep learning and efficient feature representation learning on non-Euclidean data, specifically designed for learning on sequential data with inherent topological connections. We have achieved promising experimental results on human connectome data as well as human action recognition, indicating great applicability in real-world data for neuroscience and computer vision.

References

Afsari, B.: Riemannian L p center of mass: existence, uniqueness, and convexity. Proceedings of the American Mathematical Society 139 (2), 655-673 (2011)

Anirudh, R., Turaga, P., Su, J., Srivastava, A.: Elastic functional coding of human actions: From vector-fields to latent variables. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3147-3155 (2015)

Bassett, D.S., Sporns, O.: Network neuroscience. Nature Neuroscience 20 , 353-364 (2017)

Bassett, D.S., Wymbs, N.F., Porter, M.A., Mucha, P.J., Carlson, J.M., Grafton, S.T.: Dynamic reconfiguration of human brain networks during learning. Proceedings of the National Academy of Sciences 108 (18), 7641-7646 (2011)

Ben Tanfous, A., Drira, H., Ben Amor, B.: Coding kendall's shape trajectories for 3d action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2840-2849 (2018)

Bilinski, P., Bremond, F.: Video covariance matrix logarithm for human action recognition in videos. In: IJCAI 2015-24th International Joint Conference on Artificial Intelligence (IJCAI) (2015)

Bouritsas, G., Frasca, F., Zafeiriou, S., Bronstein, M.M.: Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (1), 657-668 (2022)

Breakspear, M.: Dynamic models of large-scale brain activity. Nature neuroscience 20 (3), 340-352 (2017)

Cai, H., Dan, T., Huang, Z., Wu, G.: Osr-net: Ordinary differential equation-based brain state recognition neural network. In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). pp. 1-5. IEEE (2023)

Chakraborty, R., Yang, C.H., Zhen, X., Banerjee, M., Archer, D., Vaillancourt, D., Singh, V., Vemuri, B.: A statistical recurrent model on the manifold of symmetric positive definite matrices. Advances in Neural Information Processing Systems 31 (2018)

Cherian, A., Sra, S., Banerjee, A., Papanikolopoulos, N.: Efficient similarity search for covariance matrices via the jensen-bregman logdet divergence. In: 2011 International Conference on Computer Vision. pp. 2399-2406. IEEE (2011)

Chiesa, P.A., Cavedo, E., Lista, S., Thompson, P.M., Hampel, H.: Revolution of restingstate functional neuroimaging genetics in alzheimer's disease. Trends in Neurosciences 40 (8), 469-480 (2017)

Chikuse, Y.: State space models on special manifolds. Journal of Multivariate Analysis 97 (6), 1284-1294 (2006)

Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)

Dan, T., Huang, Z., Cai, H., Laurienti, P.J., Wu, G.: Learning brain dynamics of evolving manifold functional mri data using geometric-attention neural network. IEEE Transactions on Medical Imaging 41 (10), 2752-2763 (2022)

Dan, T., Huang, Z., Cai, H., Lyday, R.G., Laurienti, P.J., Wu, G.: Uncovering shape signatures of resting-state functional connectivity by geometric deep learning on riemannian manifold. Human Brain Mapping 43 (13), 3970-3986 (2022)

Dan, T., Wei, Z., Kim, W.H., Wu, G.: Exploring the enigma of neural dynamics through a scattering-transform mixer landscape for Riemannian manifold. In: Proceedings of the 41st International Conference on Machine Learning. vol. 235, pp. 9976-9990 (2024)

Destrieux, C., Fischl, B., Dale, A., Halgren, E.: Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. NeuroImage 53 (1), 1-15 (2010)

Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., Del Bimbo, A.: 3-d human action recognition by shape analysis of motion trajectories on riemannian manifold. IEEE Transactions on Cybernetics 45 (7), 1340-1352 (2014)

Di Martino, A., Yan, C.G., Li, Q., Denio, E., Castellanos, F.X., Alaerts, K., Anderson, J.S., Assaf, M., Bookheimer, S.Y., Dapretto, M., et al.: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Molecular Psychiatry 19 , 659-667 (2014)

Ding, J., Dan, T., Wei, Z., Cho, H., Laurienti, P.J., Kim, W.H., Wu, G.: Machine learning on dynamic functional connectivity: Promise, pitfalls, and interpretations. arXiv preprint arXiv:2409.11377 (2024)

Faisan, S., Thoraval, L., Armspach, J.P., Heitz, F.: Hidden markov multiple event sequence models: a paradigm for the spatio-temporal analysis of fmri data. Medical Image Analysis 11 (1), 1-20 (2007)

Gao, X., Hu, W., Tang, J., Liu, J., Guo, Z.: Optimized skeleton-based action recognition via sparsified graph regression. In: Proceedings of the 27th ACM international conference on multimedia. pp. 601-610 (2019)

Glasser, M.F., Coalson, T.S., Robinson, E.C., Hacker, C.D., Harwell, J., Yacoub, E., Ugurbil, K., Andersson, J., Beckmann, C.F., Jenkinson, M., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536 , 171-178 (2016)

Gu, A., Dao, T.: Mamba: Linear-time sequence modeling with selective state spaces. In: First Conference on Language Modeling (2024)

Gu, A., Goel, K., Gupta, A., Ré, C.: On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems 35 , 35971-35983 (2022)

Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T., Rudra, A., Ré, C.: Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in Neural Information Processing Systems 34 , 572-585 (2021)

Guo, K., Ishwar, P., Konrad, J.: Action recognition from video using feature covariance matrices. IEEE Transactions on Image Processing 22 (6), 2479-2494 (2013)

Han, K., Yang, Y., Huang, Z., Kan, X., Guo, Y., Yang, Y., He, L., Zhan, L., Sun, Y., Wang, W., et al.: Brainode: Dynamic brain signal analysis via graph-aided neural ordinary differential equations. In: 2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI). pp. 1-8. IEEE (2024)

Hasani, R., Lechner, M., Amini, A., Rus, D., Grosu, R.: Liquid time-constant networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 7657-7666 (2021)

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9 (8), 1735-1780 (1997)

Huang, Z., Van Gool, L.: A riemannian network for spd matrix learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 31 (2017)

Huang, Z., Cai, H., Dan, T., Lin, Y., Laurienti, P., Wu, G.: Detecting brain state changes by geometric deep learning of functional dynamics on riemannian manifold. In: Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VII 24. pp. 543-552. Springer (2021)

Hussein, M.E., Torki, M., Gowayyed, M.A., El-Saban, M.: Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations. In: Twenty-third International Joint Conference on Artificial Intelligence (2013)

Hutchinson, R.A., Niculescu, R.S., Keller, T.A., Rustandi, I., Mitchell, T.M.: Modeling fmri data generated by overlapping cognitive processes with unknown onsets using hidden process models. NeuroImage 46 (1), 87-104 (2009)

Hutchison, R.M., Womelsdorf, T., Allen, E.A., Bandettini, P.A., Calhoun, V.D., Corbetta, M., Della Penna, S., Duyn, J.H., Glover, G.H., Gonzalez-Castillo, J., et al.: Dynamic functional connectivity: promise, issues, and interpretations. NeuroImage 80 , 360-378 (2013)

Jeong, S., Ko, W., Mulyadi, A.W., Suk, H.I.: Deep efficient continuous manifold learning for time series modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (1), 171-184 (2023)

Just, M.A., Cherkassky, V.L., Keller, T.A., Minshew, N.J.: Cortical activation and synchronization during sentence comprehension in high-functioning autism: evidence of underconnectivity. Brain 127 (8), 1811-1821 (2004)

Kalman, R.E.: A new approach to linear filtering and prediction problems. Journal of Basic Engineering 82 (1), 35-45 (1960)

Kan, X., Dai, W., Cui, H., Zhang, Z., Guo, Y., Yang, C.: Brain network transformer. Advances in Neural Information Processing Systems 35 , 25586-25599 (2022)

Kanatsoulis, C., Ribeiro, A.: Counting graph substructures with graph neural networks. In: The Twelfth International Conference on Learning Representations (2024)

Karahano˘ glu, F.I., Van De Ville, D.: Dynamics of large-scale fmri networks: Deconstruct brain activity to build better models of brain function. Current Opinion in Biomedical Engineering 3 , 28-36 (2017)

Kim, B.H., Ye, J.C., Kim, J.J.: Learning dynamic graph representation of brain connectome with spatio-temporal attention. In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems (2021)

Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (2017)

Koniusz, P., Cherian, A., Porikli, F.: Tensor representations via kernel linearization for action recognition from 3d skeletons. In: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14. pp. 37-53. Springer (2016)

LaMontagne, P.J., Benzinger, T.L., Morris, J.C., Keefe, S., Hornbeck, R., Xiong, C., Grant, E., Hassenstab, J., Moulder, K., Vlassenko, A.G., et al.: Oasis-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease. medrxiv pp. 2019-12 (2019)

Li, H., Fan, Y.: Identification of temporal transition of functional states using recurrent neural networks from functional mri. In: Medical Image Computing and Computer Assisted Intervention-MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part III 11. pp. 232-239. Springer (2018)

Li, H., Fan, Y.: Interpretable, highly accurate brain decoding of subtly distinct brain states from functional mri using intrinsic functional networks and long short-term memory recurrent neural networks. NeuroImage 202 , 116059 (2019)

Li, X., Zhou, Y., Dvornek, N., Zhang, M., Gao, S., Zhuang, J., Scheinost, D., Staib, L.H., Ventola, P., Duncan, J.S.: Braingnn: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis 74 , 102233 (2021)

Marek, K., Jennings, D., Lasch, S., Siderowf, A., Tanner, C., Simuni, T., Coffey, C., Kieburtz, K., Flagg, E., Chowdhury, S., et al.: The parkinson progression marker initiative (ppmi). Progress in Neurobiology 95 (4), 629-635 (2011)

Menon, V.: Large-scale brain networks and psychopathology: a unifying triple network model. Trends in Cognitive Sciences 15 (10), 483-506 (2011)

Mueller, S.G., Weiner, M.W., Thal, L.J., Petersen, R.C., Jack, C., Jagust, W., Trojanowski, J.Q., Toga, A.W., Beckett, L.: The alzheimer's disease neuroimaging initiative. Neuroimaging Clinics of North America 15 (4), 869-877 (2005)

Müller, M., Röder, T., Clausen, M., Eberhardt, B., Krüger, B., Weber, A.: Mocap database hdm05. Institut für Informatik II, Universität Bonn 2 (7) (2007)

Niu, H., Zhou, Y., Yan, X., Wu, J., Shen, Y ., Yi, Z., Hu, J.: On the applications of neural ordinary differential equations in medical image analysis. Artificial Intelligence Review 57 , 236 (2024)

Palop, J.J., Chin, J., Mucke, L.: A network dysfunction perspective on neurodegenerative diseases. Nature 443 , 768-773 (2006)

Paoletti, G., Cavazza, J., Beyan, C., Del Bue, A.: Subspace clustering for action recognition with covariance representations and temporal pruning. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 6035-6042. IEEE (2021)

Plub-in, N., Songsiri, J.: State-space model estimation of eeg time series for classifying active brain sources. In: 2018 11th Biomedical Engineering International Conference (BMEiCON). pp. 1-5. IEEE (2018)

Rudie, J.D., Dapretto, M.: Altered functional and structural brain network organization in autism. NeuroImage: Clinical 2 , 79-94 (2013)

Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323 , 533-536 (1986)

Said, A., Bayrak, R., Derr, T., Shabbir, M., Moyer, D., Chang, C., Koutsoukos, X.: Neurograph: Benchmarks for graph machine learning in brain connectomics. Advances in Neural Information Processing Systems 36 , 6509-6531 (2023)

Seidenari, L., Varano, V., Berretti, S., Bimbo, A., Pala, P.: Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp. 479-485 (2013)

Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: Ntu rgb+ d: A large scale dataset for 3d human activity analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1010-1019 (2016)

Singh, M.F., Braver, T.S., Cole, M.W., Ching, S.: Estimation and validation of individualized dynamic brain models with resting state fmri. NeuroImage 221 , 117046 (2020)

Suk, H.I., Wee, C.Y., Lee, S.W., Shen, D.: State-space model with deep learning for functional dynamics estimation in resting-state fmri. NeuroImage 129 , 292-307 (2016)

Tolstikhin, I.O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al.: Mlp-mixer: An all-mlp architecture for vision. Advances in Neural Information Processing Systems 34 , 24261-24272 (2021)

Tu, T., Paisley, J., Haufe, S., Sajda, P.: A state-space model for inferring effective connectivity of latent neural dynamics from simultaneous eeg/fmri. Advances in Neural Information Processing Systems 32 (2019)

Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., Joliot, M.: Automated anatomical labeling of activations in spm using a macroscopic anatomical parcellation of the mni mri single-subject brain. NeuroImage 15 (1), 273-289 (2002)

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in Neural Information Processing Systems 30 (2017)

Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3d skeletons as points in a lie group. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 588-595 (2014)

Viola, K.L., Sbarboro, J., Sureka, R., De, M., Bicca, M.A., Wang, J., Vasavada, S., Satpathy, S., Wu, S., Joshi, H., et al.: Towards non-invasive diagnostic imaging of early-stage alzheimer's disease. Nature Nanotechnology 10 (1), 91-98 (2015)

Wang, L., Zhang, J., Zhou, L., Tang, C., Li, W.: Beyond covariance: Feature representation with nonlinear kernel matrices. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 4570-4578 (2015)

Wang, P., Yuan, C., Hu, W., Li, B., Zhang, Y.: Graph based skeleton motion representation and similarity measurement for action recognition. In: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VII 14. pp. 370-385. Springer (2016)

Weng, J., Weng, C., Yuan, J.: Spatio-temporal naive-bayes nearest-neighbor (st-nbnn) for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4171-4180 (2017)

Wu, H., Lu, M., Zeng, Y.: State estimation of hemodynamic model for fmri under confounds: Ssm method. IEEE Journal of Biomedical and Health Informatics 24 (3), 804-814 (2019)

Xia, L., Chen, C.C., Aggarwal, J.K.: View invariant human action recognition using histograms of 3d joints. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. pp. 20-27. IEEE (2012)

Xu, J., Bian, Q., Li, X., Zhang, A., Ke, Y., Qiao, M., Zhang, W., Sim, W.K.J., Gulyás, B.: Contrastive graph pooling for explainable classification of brain networks. IEEE Transactions on Medical Imaging (2024)

Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018)

Yang, Y., Deng, C., Gao, S., Liu, W., Tao, D., Gao, X.: Discriminative multi-instance multitask learning for 3d action recognition. IEEE Transactions on Multimedia 19 (3), 519-529 (2016)

Zhang, J., Li, W., Wang, P., Ogunbona, P., Liu, S., Tang, C.: A large scale rgb-d dataset for action recognition. In: International workshop on understanding human activities through 3D sensors. pp. 101-114. Springer (2016)

Zhang, T., Zheng, W., Cui, Z., Zong, Y., Li, C., Zhou, X., Yang, J.: Deep manifold-to-manifold transforming network for skeleton-based action recognition. IEEE Transactions on Multimedia 22 (11), 2926-2937 (2020)

Zhao, L., Jin, W., Akoglu, L., Shah, N.: From stars to subgraphs: Uplifting any GNN with local structure awareness. In: International Conference on Learning Representations (2022)

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Please refer to Abstract.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please refer to Appendix A.10.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Please refer to Appendix A.2 to A.5.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please refer to Experiments part and Appendix A.8.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Please refer to Appendix.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please refer to Appendix A.8.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We use paired t -test, please refer to Sec. 4.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please refer to Appendix A.8.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Please refer the whole manuscript.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Please refer to Appendix A.11.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We have the code to release and have uploaded it to GitHub.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: the paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

A Appendix

A.1 Literature Survey

RNN and its variants on manifold to neuroimaging application. Recurrent neural networks (RNNs) have been reformulated as ordinary differential equations (ODEs) with continuous-time system states, as highlighted by LTCNet [30]. These models serve as effective algorithms for modeling time series data and are widely utilized across medical, industrial, and business domains. For instance, [9] has demonstrated its potential for brain state recognition and [29] achieves continuous modeling of dynamic brain signals using ODEs. Furthermore, the survey proposed by [54] provides a comprehensive overview of ODE applications in the field of medical imaging, showcasing their practicality and impact in this domain. Following this, several manifold-based RNN models have emerged. For instance, [10] introduced a statistical recurrent model defined on the manifold of symmetric positive definite (SPD) matrices and evaluated its diagnostic potential for neuroimaging applications. This approach underscores the effectiveness of utilizing manifold-based techniques to enhance the performance of RNNs in complex medical contexts. The RNN model formulated on Riemannian manifolds is robustly supported by mathematical theory, as it utilizes covariance information to dynamically model time-series data [37]. This capability allows it to capture richer and more subtle representations within a higher-dimensional latent space. Such an approach is particularly effective in modeling complex data structures, such as capturing the functional dynamics [15; 33], where the relationships among data points are inherently geometric. By operating within the manifold framework, these models adeptly accommodate the intricacies of underlying data distributions, thereby enhancing both interpretability and predictive performance.

RNNs and their variants, while widely used for sequential modeling tasks, have notable limitations that affect their performance in complex, dynamic systems. One of the key challenges is that RNNs implicitly learn sequential patterns and temporal dependencies, without explicitly modeling the underlying dynamics. This implicit nature makes RNNs harder to interpret, often turning them into 'black-box' models where the relationships between input variables and predicted outcomes can be obscured, limiting their utility in scenarios requiring high interpretability. Although advancements like LTCNet [30] have improved the interpretability of RNNs by framing them as an ODE, these models primarily focus on the dynamics of the system states and inputs (as shown in Fig. 1 (a)). However, they failed to consider observation equations (but usually use MLP to fit the observations), which describe the relationship between system states and observed data. This formulation reduces their ability to fully model the observable aspects of a system, resulting in an incomplete picture of the system's dynamics and limiting their explanatory power.

SSM to neuroimaging application. State Space Models (SSMs) explicitly model temporal dynamics through latent variables governed by two key ODEs: the state equation, which captures the evolution of the system state over time, and the observation equation, which relates the latent state to observable data. This structured, ODE-based framework allows SSMs to offer a clearer understanding of how systems evolve and provides a higher level of interpretability compared to RNNs. This makes SSMs particularly valuable in domains requiring an understanding of underlying system dynamics, such as medical diagnostics and time-series forecasting. In contrast to RNNs and their variants (e.g., LSTMs, GRUs), which often operate as 'black boxes,' SSMs like Kalman Filters [39] have well-established theoretical properties. These properties typically include convergence and stability, providing a solid mathematical foundation that is difficult to guarantee with more complex RNN architectures. RNNs, especially deeper ones, can suffer from issues like vanishing or exploding gradients, which affect training stability and interoperability. SSMs also naturally incorporate probabilistic structures, allowing them to effectively handle noisy or uncertain data. This is particularly advantageous in low Signal-to-Noise Ratio (SNR) datasets, such as fMRI [74] and Electroencephalogram (EEG) data [57], where the ability to account for noise and uncertainty is critical. In light of these performance advantages, only a few manifold-based SSMs have been developed. For instance, [13] explores the modeling of time series observations in state-space forms defined on Stiefel and Grassmann manifolds. This approach utilizes Bayesian methods to estimate state matrices by calculating posterior modes, effectively integrating geometric constraints with probabilistic inference. However, while Bayesian methods excel in handling uncertainty, they often face limitations in scalability, inference speed, and flexibility compared to deep learning models, which offer more efficient and powerful representation capabilities for large-scale data.

In this context, the introduction of deep geometric SSMs aims to combine the representational power of deep neural networks with the interpretability and structured dynamics inherent in traditional SSMs. By incorporating the geometric properties of manifold-based modeling, these models adeptly capture the intrinsic structure of the data, which is crucial for accurately representing complex relationships in high-dimensional datasets, such as those found in brain imaging. This combination not only enhances interpretability but also allows for a more nuanced understanding of the underlying dynamics, ultimately improving the efficacy of the modeling process.

A.2 Geodesic Distances Isometry

We aim to prove that the 'translation' operation defined by orthogonal transformations on the SPD manifold equipped with the Stein metric is an isometry. Specifically, we show:

Given the SPD manifold:

the translation operation for orthogonal matrices g ∈ O (where g ⊤ g = gg ⊤ = I ) is defined as:

The Stein metric is defined as:

Orthogonal matrices preserve determinants:

We explicitly verify the invariance under orthogonal transformations:

Since det( g ) = ± 1 , we have log det( g ) = 0 . Thus,

This proves the translation operation under orthogonal transformations is an isometry on the SPD manifold with the Stein metric.

A.3 SPD Convolution Operation

Proof. Since H is SPD, it can be decomposed as follows:

where Z = [ z 1 , z 2 , . . . , z θ ] is a matrix of full rank. The convolutional result of an SPD representation matrix X ∈ R N × N can then be expressed as:

and

where the transition from Eq. 26 to Eq. 27 uses the property of separable convolution. Suppose z i = [ m i 1 , m i 2 , . . . , m iθ ] ⊤ , for i = 1 , 2 , . . . , θ . The convolution between X and z i can be written as:

where P z i ∈ R ( M -N +1) × M and

Thus, the following equations hold:

̸

Since the rank of P z i equals M -N +1 , the matrix P z i XP ⊤ z i is also SPD. Therefore, for any q ∈ R M where q = 0 , we have:

Hence, O is an SPD matrix.

Furthermore, the k -th channel of X can be written as:

where X ( l ) denotes the l -th channel of the input descriptor. Since X ( l ) and H ( k,l ) are SPD matrices, and according to the above proof, X ( l ) is also an SPD matrix. Therefore, X ( k ) is a multi-channel SPD matrix.

A.4 SPD exp( · ) Operation

Proof: Since X is symmetric, we know that for any integer k , the powers X k are also symmetric. The matrix exponential of X is defined by the following power series:

Each term in this series involves a symmetric matrix X k , and the sum of symmetric matrices remains symmetric. Therefore, exp( X ) is symmetric.

Since X is symmetric, it can be diagonalized as: X = Q Λ Q ⊤ , where Q is an orthogonal matrix (i.e., Q ⊤ Q = I ) and Λ is a diagonal matrix containing the eigenvalues λ 1 , λ 2 , . . . , λ n of X . Because X is positive definite, all eigenvalues λ i are positive, i.e., λ i > 0 for all i .

The matrix exponential exp( X ) is then given by:

where exp(Λ) is the diagonal matrix with entries exp( λ 1 ) , exp( λ 2 ) , . . . , exp( λ n ) . Since the exponential function satisfies exp( λ i ) > 0 for all λ i ∈ R , each eigenvalue of exp( X ) is positive. Thus, exp( X ) has strictly positive eigenvalues, and since it is symmetric, it is also positive definite.

A.5 SPD Padding [ · ] Operation

Given a SPD matrices X ∈ Sym + N and a small positive value ρ , the assemble matrix Y = diag ( ρ, X ) = [ ρ 0 0 X ] is a SPD matrix.

Proof: First, Y is a symmetric, since Y ⊤ = [ ρ 0 0 X ] ⊤ = [ ρ 0 0 X ] = Y . Then, to show that Y is positive definite, we need to verify that for any non-zero vector z = [ z 1 z 2 ] ∈ R N +1 , the quadratic form z ⊤ Y z is strictly positive.

We compute the quadratic form:

̸

Since ρ > 0 , the term z 2 1 ρ ≥ 0 , and it is strictly positive if z 1 = 0 .

Furthermore, since X ∈ Sym + N , X is positive definite, meaning z ⊤ 2 Xz 2 > 0 for any non-zero z 2 ∈ R N .

Thus, for any non-zero vector z = [ z 1 z 2 ]

which proves Y ∈ Sym + N +1 is a SPD matrix.

A.6 Dataset

For HAR dataset. We evaluate the performance of the proposed GeoDynamics on three benchmark HAR datasets: the Florence 3D Actions dataset [61], the HDM05 database [53], and the UTKinectAction3D (UTK) dataset [75]. The Florence 3D Actions dataset includes 9 activities ( answer phone, bow, clap, drink, read watch, sit down, stand up, tie lace, wave ), performed by 10 subjects, with each activity repeated 2 to 3 times, resulting in a total of 215 samples. These actions are represented by the motion of 15 skeletal joints. For the HDM05 dataset, we follow the protocol outlined in [71], selecting 14 action classes ( clap above head, deposit floor, elbow to knee, grab high, hop both legs, jog, kick forward, lie down on floor, rotate both arms backward, sit down chair, sneak, squat, stand up, throw basketball ). The sequences, captured using VICON cameras, result in 686 samples, each represented by 31 skeletal joints-significantly more than in the Florence dataset. The increased number of joints and higher intra-class variability make this dataset particularly challenging. Finally, the UTKinect-Action3D dataset consists of 10 action classes ( carry, clap hands, pick up, pull, push, sit down, stand up, throw, walk, wave hands ), captured using a stationary Microsoft Kinect camera. Each action was performed twice by 10 subjects, yielding 199 samples in total.

For HBC dataset. We select one dataset of healthy young adults and four disease-related human brain datasets for evaluation: the Human Connectome Project-Young Adult Working Memory (HCPWM)[79], Alzheimer's Disease Neuroimaging Initiative (ADNI) [52], Open Access Series of Imaging

Studies (OASIS) [46], Parkinson's Progression Markers Initiative (PPMI) [50], and the Autism Brain Imaging Data Exchange (ABIDE). We selected a total of 1,081 subjects from the HCP-WM dataset. The working memory task included both 2-back and 0-back conditions, with stimuli featuring images of bodies, places, faces, and tools, interspersed with fixation periods. The specific task events are: 2bk-body, 0bk-face, 2bk-tool, 0bk-body, 0bk-place, 2bk-face, 0bk-tool, and 2bk-place. Brain activity was parcellated into 360 regions based on the multi-modal parcellation from [24]. For the OASIS (924 subjects) and ABIDE (1,025 subjects) datasets, which are binary-class datasets, one class represents a disease group and the other represent healthy controls. In the ADNI dataset, subjects are categorized based on clinical outcomes into distinct cognitive status groups: cognitively normal (CN), subjective memory concern (SMC), early-stage mild cognitive impairment (EMCI), late-stage mild cognitive impairment (LMCI), and Alzheimer's Disease (AD). For population analysis, we group CN, SMC, and EMCI into a 'CN-like" group, while LMCI and AD form the 'AD-like" group. This grouping enables a detailed analysis of cognitive decline and disease progression. The PPMI dataset consists of four classes: normal control, scans without evidence of dopaminergic deficit (SWEDD), prodromal Parkinson's disease, and Parkinson's disease (PD). This classification supports the study of different stages of Parkinson's progression. We employ Automated Anatomical Labeling (AAL) atlas [67] (116 brain regions) on ADNI, PPMI, ABIDE datasets, while Destrieux atlas [18] (160 brain regions) is used in OASIS to verify the scalability of the models.

A.7 Comparison Methods and Experimental Results

We roughly summarize the comparison methods for HBC into two categories: spatial models and sequential models.

Spatial models. The spatial models are essential for understanding brain dynamics. Traditional GNNs like graph convolutional network (GCN) [44] and graph isomorphism network (GIN) [77] are selected for their ability to effectively capture diffusion patterns and isomorphism encoding in structured data. Subgraph-based GNNs, such as Moment-GNN [41], emphasize subgraph structures, enabling the identification of localized patterns that might be overlooked by traditional GNNs. Expressive GNNs, including graph substructure network (GSN) [7] and GNNAsKernel (GNN-AK) [81], are chosen for their enhanced expressivity through subgraph isomorphism counting and local subgraph encoding, which could be crucial for distinguishing subtle differences in complex systems.

Amanifold-based model like the symmetric positive definite network (SPDNet) [16] is adopted for its ability to manage high-dimensional manifold data, making it suitable for more complicated datasets.

Two graph-based brain network analysis models for disease diagnosis, BrainGNN [49], an interpretable brain graph neural network for fMRI analysis, and ContrastPool [76], a contrastive dual-attention block and a differentiable graph pooling method.

Additionally, a traditional multi-layer perceptron (MLP) serves as a model due to its efficiency and versatility across various domains.

For all spatial models, following the optimal settings described in [60], we use the vectorized static functional connectivity (FC) as graph embeddings and the static FC matrices ( N × N ) as adjacency matrices, where only the top 10% of edges are retained through thresholding to ensure sparsity. The input of SPDNet is the original N × N FC matrices.

For dynamic-FC models (STAGIN [43] and NeuroGraph [60], the thresholded dynamic FC matrices serve as the graph, NeuroGraph serve the vectorized FC as the embedding and STAGIN incorporates BOLD signals as part of the embedding, alongside its unique embedding construction method. For our GeoDynamics , we use the dynamic FC matrices as the input, resulting in T × N × N matrices.

Sequential models. The sequential models are selected to analyze temporal dynamics in time-series BOLD signals. 1D-CNN is chosen for its ability to capture temporal patterns through convolutional operations. RNN [59] and LSTM [31] are included for their proficiency in modeling sequential data and capturing long-range dependencies. MLP-Mixer [65] is selected for its capability to mix both temporal and spatial features, offering a comprehensive view by integrating information across different dimensions. Transformer [68] is chosen for its powerful attention mechanisms, which allow it to capture global dependencies in sequential data. Brain network transformer (BNT) [40] is a tailored approach specifically designed for brain network analysis. Lastly, the state-space model

(SSM), represented by Mamba [25], is selected for its advanced state-space modeling abilities that effectively capture system dynamics over time.

For the sequential models, the inputs are the BOLD signals ( N × T ).

Note, the inputs for all comparison methods align with the recent work presented in [21], ensuring fairness in the evaluation process.

We further conducted experiments using three brain network analysis models on disease-based datasets, including ADNI, OASIS, PPMI, and ABIDE. The diagnostic accuracies of 10-fold crossvalidation are presented in Table 5. It is clear that our GeoDynamics consistently outperforms all the compared methods.

A.8 Hyperparameters, Inference time and Number of Parameters

We list the detailed hyperparameters of each model in Table 6. We summarize the inference time and the number of parameters of each mode on HCP-WM dataset ( N = 360 , T = 39 ) in Table 7, all the experiments are conducted on NVIDIA RTX 6000Ada GPUs. We can observe that our method efficiently utilized the parameters compared to most counterpart methods. Compared to Mamba (vanilla SSM), our method requires more time in the final step due to the logarithmic mapping, which involves the computationally expensive Singular Value Decomposition (SVD). However, it is more efficient than SPDNet (a manifold-based model), as we leverage convolution operations. As a result, the overall computational cost remains manageable.

A.9 Discussion

We expect our manifold-based deep model to facilitate our understanding on brain behavior in the following ways.

(1) Enhance the prediction accuracy. A plethora of neuroscience findings indicate that fluctuation of functional connectivities exhibits self-organized spatial-temporal patterns. Following this notion, we conceptualize that well-defined mathematical modeling of intrinsic data geometry of evolving functional connectivity (FC) matrices might be the gateway to enhance prediction accuracy. Our experiments have shown that respecting the intrinsic data geometry in method development leads to significantly higher prediction accuracy for cognitive states, as demonstrated in Fig. 3.

(2) Enhance the model explainability. We train the deep model to parameterize the transition of FC matrices on the Riemannian manifold (Eq. 14). By doing so, we are able to analyze the temporal behaviors with respect to each cognitive state using post-hoc complex system approaches such as dynamic mode decomposition, stability analysis.

(3) Provide a high-order geometric attention mechanism that is beyond node-wise or link-wise focal patterns. Conventional methods often employ attention components for each region or link in the brain network separately, thus lacking the high-order attention maps associated with neural circuits (i.e., a set of links representing a sub-network). In contrast, the SPD-preserving attention mechanism (Eq. 16) in our method operates on the Riemannian manifold, taking the entire brain network into account. As shown in Fig. 4, our method has identified not only links but also sub-networks relevant to cognitive states and disease outcomes.

A.10 Limitations and Future Work

First, the use of matrix exponentials and logarithmic operations (core to our Riemannian framework), requires eigenvalue or Cholesky decompositions, which are computationally intensive. This leads to higher complexity compared to standard Euclidean approaches, especially as the number of nodes (i.e., brain regions) increases. To address this challenge, we can leverage parallel computing to mitigate the computational burden.

Second, the effectiveness of manifold-based modeling depends on the assumption that input SPD matrices lie within a well-behaved region of the manifold (e.g., within a geodesic ball), which ensures the uniqueness and stability of the Fréchet mean. When this assumption is violated, such as in highly noisy or poorly conditioned data (rank-deficient), the performance of the method may degrade.

Third, interpretability remains a challenge. While the geometric framework captures intrinsic data structure more faithfully, understanding how specific patterns relate to clinical or cognitive outcomes is still an open area of research.

In future work, we plan to: (1) Optimize runtime further through low-rank approximations or manifoldaware pruning. (2) Extend the framework to handle multi-modal neuroimaging data (e.g., EEG, MEG). (3) Explore other interpretable models to bridge the gap between mathematical complexity and clinical insight.

A.11 Impact Statement

This work bridges the fields of machine learning and neuroscience by introducing a geometric deep learning-based state-space model ( GeoDynamics ) for understanding brain dynamics and their relationship to behavior and cognition. By leveraging the unique properties of Riemannian geometry, our model offers a holistic view of brain state evolution as a self-organized dynamical system, addressing critical challenges in functional neuroimaging studies and enabling applications in early diagnosis of neurodegenerative diseases such as Alzheimer's and Parkinson's.

Beyond neuroscience, GeoDynamics demonstrates broad applicability in capturing spatio-temporal dynamics across diverse domains, as evidenced by its promising performance in human action recognition benchmarks. This highlights the potential of our approach to impact fields ranging from healthcare to computer vision, offering tools for scalable, robust, and interpretable analysis of complex systems.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a neural network based on a state-space model that operates on the Riemannian manifold of SPD matrices. The proposed network is tailored for processing sequences of functional connectivity matrices, which are modeled as SPD matrices that capture the dynamics of brain activity. The proposed network is evaluated on various benchmarks involving brain dynamics and action recognition.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The addressed problems in neuroscience are important and central\n- The proposed network is new and nicely combines concepts and tools from different fields, where the internal state and output updates are performed “on the manifold” using translation and the weighted Riemannian mean (Eq. 3)\n- The experiments demonstrate the benefits of the proposed network compared to baselines in a very broad range of applications\n\nWeaknesses:\n- The presentation and writing need some improvement, specifically, the treatment of the Riemannian geometry of SPD matrices (see details below)\n- Implementations details are missing - for example, details on how the Frechet mean is computed\n- The computational complexity is mentioned only in Appendix A.11 in the context of limitations without details. From practical standpoint, this is an important issue that should be better discussed. Specifically, what is the computational complexity of the algorithm and how does it scale with the size of the FC matrices?\n\nQUESTIONS:\nSpecific comments and suggestions:\n- Line 74: specify which “Riemannian manifold”\n- Line 80: “manifold instance” => sample/point on the manifold\n- Sec. 2.2 - “Riemannian geometry algebra” => “Riemannian geometry”\n- Sec. 2.2, line 120: the notation in parenthesis is not clear. My suggestion is to replace \\mathcal{M} => sym^+_N.\n- Sec. 2.2 - only the Stein metric (aka Jensen-Bregman logdet divergence) [10] is mentioned and used but there are many other Riemanian metrics on the SPD manifold, which have been proposed and studied extensively. It is important to mention the alternatives, e.g., the affine-invariant metric [*] and log Euclidean [**], and discuss why the particular logdet divergence was chosen.\n\n[*] Pennec, X., Fillard, P., & Ayache, N. (2006). A Riemannian framework for tensor computing. International Journal of computer vision, 66, 41-66.\n\n[**] Arsigny, V., Fillard, P., Pennec, X., & Ayache, N. (2006). Log‐Euclidean metrics for fast and simple calculus on diffusion tensors. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 56(2), 411-421.\n\nLIMITATIONS:\nSome limitations are mentioned in the appendix (A.11) but I find it quite hidden, and some of the discussion could be moved to the main text", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe goal of this paper is to analyze functional MRI (fMRI) data to study human brain with respect to clinical conditions or some cognitive tasks being performed by the patient. There are two claimed innovations in this paper. Firstly, it uses a state-space model on the SPDM manifold to model the times series data. Specifically, the approach is to transform the data into a time series of brain connectivity matrices, and to model it as a time series on SPDM manifold. To analyze this time series, the paper formulates a Markov decision process that models a hidden state, a transition matrix, and an output that classifies the state. The paper user geometry of SPDM to formulate these elements intrinsically on the manifold. Secondly, the inference is performed using a neural-network architecture. It defines a convolution operation on SPDMs for use in forming convolution layers in the network. \n\nThere is a very extensive experimental support to the approach in the paper. It compares several methods on datasets for classifying cognitive tasks and neurodegenerative diseases from fMRI data. The results are generally superior to the existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1.\tIt utilizes the geometry of SPDMs to respect the structure of brain connectivity. Several papers have utilized this representation to constrain analysis to interpretable solutions. \n2.\tIt implements a deep-learning version of the state space models on nonlinear manifolds. \n3.\tThe experimental results are both extensive and impressive. \n\nWeaknesses: \n\n1.\tThe modeling of brain functional connectivity using a time-series of SPD matrices is not new, a number of papers have used this formulation. See for example, Dai et al, Analyzing Dynamical Brain Connectivity as Trajectories on Space of Covariance Matrices, 2020. Implementing this SSM using neural networks has some novelty to it. \n2.\tThe computational cost of performing analysis on SPDM manifold is expensive. The matrix computations grow much faster than the size of matrices. The method will not scale as well if the number of nodes in the brain increases. \n3.\tEven though the empirical results are impressive, this formulation is hard to analyze theoretically due to nonlinearity of the underlying manifold.\n\nQUESTIONS:\nPlease clarify the novelty of your approach to the part papers on dynamic brain functional connectivity.\n\nLIMITATIONS:\nI don't see any note on the limitations of the model/method proposed here.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces GeoDynamics, a geometric state-space neural network designed to model time-varying functional connectivity (FC) in the brain. Unlike standard SSMs which operate in Euclidean space, GeoDynamics constrains both system inputs, hidden states, and outputs to lie on the Symmetric Positive Definite (SPD) manifold, motivated by the observation that brain FC matrices are naturally SPD. The transition dynamics are defined using group actions (particularly of the orthogonal group) and Riemannian constructs like the Fréchet mean and the Stein metric. The model is evaluated on both human brain datasets and a human action recognition benchmark.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- Principled use of Riemannian geometry: The authors leverage interesting manifold-aware constructs (e.g., the Fréchet mean, group actions, Stein metric) to build the model in a way that respects the geometry of SPD matrices.\n- Theoretical grounding: There are rigorous definitions and group-theoretic justifications, and the paper includes formal results and claims with proofs in the appendix.\n- Potentially generalizable: While aimed at brain data, the method could be extended to any domain where SPD-valued time series arise.\n\n### Weaknesses\n\n- Major clarity issues throughout:\n    - Core concepts such as system input, hidden state, and output are not clearly mapped to the neuroscience application (e.g., is the FC the input or the hidden state?).\n    - Equations (e.g., Equation 3) are difficult to parse and are not explained with sufficient care.\n    - Basic elements of the SSM formalism (e.g., matrices A, B, C, D) are not introduced for readers unfamiliar with control theory.\n    - Redundant or imprecise definitions (e.g., repeated definitions of group actions) obscure the main argument.\n- Lack of focus in narrative: The inclusion of the human action recognition dataset seems like an afterthought and dilutes the focus of the paper, which is ostensibly about modeling brain dynamics.\n- Experimental presentation is weak: The structure of the experimental section is muddled. Results are intermixed with speculative interpretation (”Remarks”), and important clarifications (e.g., what is being predicted, what the task is) are missing.\n- Terminological inconsistency: Terms like \"system state\" vs. \"hidden state\" are used interchangeably, it would help to stick to one of them.\n- Foundational confusion: Some parts of the model formulation appear self-inconsistent (e.g., the domain of the group action $\\mathcal{T}$ vs. its usage in Equation 3), and the writing does not help to disambiguate.\n\nQUESTIONS:\n1. Model components: Could you explicitly define what constitutes the input $X(t)$, the hidden state $S(t)$, and the observed output $Y(t)$ in the context of your neuroscience application? It is currently unclear how the functional connectivity matrices relate to these quantities.\n2. Equation 3 confusion: What is the set over which you are computing the Fréchet mean in Equation 3? $S^{(k-1)}$ appears to be a single SPD matrix, not a set. What is $\\tilde{A}$? Ostensibly it is a set of weights (scalars).\n3. Group action inconsistency: You define the translation $\\mathcal{T}$ using the action of the orthogonal group $\\mathbb{O}$ on the SPD manifold, but then apply it to the Fréchet mean of SPD matrices. If the argument to $\\mathcal{T}$ must be orthogonal, this would be undefined unless the mean is trivially the identity. Can you clarify?\n4. On Fréchet mean and uniqueness: What are the guarantees on the existence and uniqueness of the Fréchet mean under your assumptions? Do you enforce closeness on the manifold during training?\n5. Experimental focus: Why include human action recognition? How does this task align with your main objective of modeling brain dynamics? Could the model be evaluated more thoroughly on neuroscience-relevant tasks?\n\nMy score could increase if the authors:\n\n- Sharply revise the manuscript to clarify definitions and model structure,\n- Cleanly separate results from interpretation,\n- Remove redundant text and clarify inconsistent terminology,\n- Provide more focused experimental evaluation and justification for each task.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The authors designed a geometric neural state-space neural networks (GeoDynamics) to model the functional connectivity (FC) among different brain areas. They benchmarked their model against fifteen other models (sequential and spatial types of model) and two datasets (HBC and HAR).

STRENGTHS AND WEAKNESSES:
The benchmark against other models and coverage of datasets are comprehensive.

The presentation of Figures/Tables are clear.

My main concern is the improvement of current model over previous ones. In L325, the author said that "Our proposed GeoDynamics demonstrates a much better performance in these tasks". I believe they mean ADNI, OASIS and PPMI datasets. Are those improvement really big enoguh? For example, the Accuracy increase from 80.4 to 81.2 in ADNI (+1%), from 89.26 to 89.6 in OASIS (0.38%), and from 73.31 to 71.35 in PPMI (worse than previous models). What are the hyper-parameters of GeoDynamics?

Furthermore, are those values (Acc, Pre, and F1) shown in previous models copied from original papers or the authors reproduced previous models by themself?

QUESTIONS:
L44 and Figure 1b legend: Does the "black solid box" indicate Euclidean space or Manifold space?

L251: So it is a 1 out 8 classification task?
 
Where is the Discussion of this paper? Section 4.3 is not the real Discussion.

LIMITATIONS:
Yes (in supplementary)
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03