Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data
Authors: Manuel Brenner
Elias Weber
Georgia Koppe
Daniel Durstewitz
Venue: ICLR 2025 Poster (2025)
Keywords: dynamical systems
recurrent neural networks
hierarchical modelling
time series
interpretability
Nonlinear Dynamics
foundation models
Primary area: learning on time series and dynamical systems
TL;DR: A general framework for inferring dynamical systems models across multiple domains/subjects.

Abstract: In science, we are often interested in obtaining a generative model of the underlying system dynamics from observed time series. While powerful methods for dynamical systems reconstruction (DSR) exist when data come from a single domain, how to best integrate data from multiple dynamical regimes and leverage it for generalization is still an open question. This becomes particularly important when individual time series are short, and group-level information may help to fill in for gaps in single-domain data. Here we introduce a hierarchical framework that enables to harvest group-level (multi-domain) information while retaining all single-domain characteristics, and showcase it on popular DSR benchmarks, as well as on neuroscience and medical data. In addition to faithful reconstruction of all individual dynamical regimes, our unsupervised methodology discovers common low-dimensional feature spaces in which datasets with similar dynamics cluster. The features spanning these spaces were further dynamically highly interpretable, surprisingly in often linear relation to control parameters that govern the dynamics of the underlying system. Finally, we illustrate transfer learning and generalization to new parameter regimes, paving the way toward DSR foundation models.

Published as a conference paper at ICLR 2025
LEARNING INTERPRETABLE HIERARCHICAL DYNAMI- CAL SYSTEMS MODELS FROM TIME SERIES DATA
Manuel Brenner! Elias Weber Georgia Koppe Daniel Durstewitz 1,2,
Dept. of Theoretical Neuroscience, Central Institute of Mental Health (CIMH) Innteriisciplinary Center for Scientific Computing, Heidelberg University Hector Institute for Al in Psychiatry and Dept. of Psychiatry and Psychotherapy, CIMH (manuel.. wrebbeeerr.... daniel. durst ) @ hannmmnnheeiee im. de
These authors contributed equally.
ABSTRACT
In science, we are often interested in obtaining generative model of the under- lying system dynamics from observed time series. While powerful methods for dynamical systems reconstruction (DSR) exist when data come from a single do- main, how to best integrate data from multiple dynamical regimes and leverage it for generalization is still open question. This becomes particularly important when individual time series are short, and group-level information may help to fill in for gaps in single-domain data. Here introduce hierarchical framework that enables to harvest group-level (multi- doalii) information while retaining all single-domain characteristics, and showcase it on popular DSR benchmarks, well as on neuroscience and medical data. In addition to faithful reconstruction of all individual dynamical regimes, our unsupervised methodology discovers common low-dimensional feature spaces in which datasets with similar dynamics cluster. The features spanning these spaces were further dynamically highly interpretable, surprisingly in often linear relation to control parameters that govern the dynamics of the underlying system. Finally, we illustrate transfer learning and generalization to new parameter regimes, paving the way toward DSR foundation models.
INTRODUCTION
In scientific and medical applications, beyond mere time series forecasting, we are often interested in interpretable and mathematically tractable models that can be used to obtain mechanistic insights into the underlying system dynamics. Such models need to be generative in a dynamical systems sense (Durstewitz et al., 2023), i.e., when simulated, should produce dynamical behavior with the same long-term statistics as the observed system, a property that conventional time series models often lack (Fig. 29, Gilpin (2024)). This is called dynamical systems (DS) reconstruction (DSR), currently burgeoning field in scientific ML (reviewed in Durstewitz et al. (2023); Gilpin (2024)). Accounting for the fact that different time series may be sampled from fundamentally different dynamical regimes, iee with different state space topology and attractor structure, may furthermore endow such models with the ability to transfer knowledge and generalize (Göring et al., 2024). This is particularly important in scientific areas, where data is not only expensive to collect, hence sparse, but also subject significant variability (Meyer-Lindenberg, 2023; Fechtelpeter et al., 2024). The challenge in these settings is to develop models that can still identify and exploit commonalities across different datasets while being flexible enough to account for individual differences in dynamical regimes.
Hierarchical models provide a natural solution by separating domain-general general and domain-specitie features. Such models facilitate the transfer of levep information across individual datasets (Pan and Yang, 2010), avoiding overfitting and improving generalization to unobserved conditions with minimal data (Kirchmeyer et al., 2022). They further explicitly represent inter-domain variability, as often crucial in addressing scientific hypotheses, by unsupervised extraction of common feature space (Zeidman et al., 2019). While hierarchical modeling is a meanwhile established approach in statistics, including time series analysis (Berliner. 1996; Hyndman al., 201 1), it has far hardly been applied to the more challenging problem of DSR where we aim for generative models of the system dynamics that can be used for downstream analysis and simulation.

---

Published as a conference paper at ICLR 2025
In this work, we propose such a general framework for extracting hierarchical DSR models from multi- domain time series data. Our approach combines now-dimensional, system- or subject-specific feature vectors with high- digh-simensionl,, group-level parameters to produce domain-specific DSR models, which can be trained end-to-end using state-of-the techniques designed for DSR (Mikhaeil et al., 2022; Hess et al., 2023). We demonstrate the effectiveness of our method in transfer and few-shot learning of DS, where generalization to previously unobserved dynamical regimes becomes possible with minimal new data. More importantly, we find that the low-dimensional subject-specific features learn to represent crucial control parameters of the underlying DS, rendering the subject feature spaces dynamically highly interpretable. We then demonstrate how unsupervised classification in these spaces based on DS features profoundly outperforms other unsupervised methods that harvest more 'conventional" time series features, providing a strong point for the DSR perspective on time series analysis.
RELATED WORK
Dynamical Systems Reconstruction (DSR) DSR is a rapidly growing field in scientific ML. DSR models may be considered special class of time series (TS) models that, beyond mere TS prediction, aim to learn surrogate models of the hata-generatinn process from TS observations. A proper DSR model needs to preserve the long-term temporal and geometrical properties of the original DS (Fig. 29), i.e. its vector field topology and attractor structure, which then enables further scientific analysis (Brunton and Kutz, 2019; Durstewitz al., 2023; Platt et al., 2023; Gilpin, 2024). A variety of DSR methods have been advanced in recent years, either based on predefined functioon libraries and symbolic regression, such as Sparse Identification of Nonlinear Dynamics (SINDy; Brunton et al. (2016); Kaiser et al. (2018)), based on Koopman operator theory (Azencot et al., 2020; Brunton et al., 2021; Naiman and Azencot, 2021), on universal approximators such as recurrent neural networks (RNNs; Gajamannage et al. (2023); Brenner et al. (2022); Hess et al. (2023)), reservoir computing (RC; Pathak et al. (2017); Platt et al. (2022; 2023)), switching or decomposed linear DS (Linderman al., 2017; Mudrik et al., 2024a), neural ordinary differential equations (Neural ODEs; Chen et al. (2018); Ko et al. (2023)), or on methods that sit somewhere in between universal and domain-specific, like physics-informed neural networks (PINNs; Raissi al. (2019)). PINNs, like library-based methods, require sufficient domain knowledge to work well in practice (Fotiadis et al., 2023; Subramanian and Mahadevan, 2023; Mouli et al., 2023). To achieve proper reconstruction of a system's 's long-term statistics and attractor geometry, often special, control-theore techniques such as sparse or generalized teacher forcing (Brenner al., 2022; Mikhaeil et al., 2022: Hess et al., 2023) or particular regularization terms that enforce invariant measures (Platt et al., 2023; Jiang et al., 2023; Schiff et al., 2024) are used. Models using these training techniques currently represent the state of the art in this field.
training
Hierarchical Time Series and DSR Modeling Hierarchical models for representing multi-level dependencies or nested groups have a long history in statistics and machine learning (Laird and Ware, 1982; Goldstein, 1987; Gelman and Hill, 2006; McCulloch et al., 201 In the time series domain, Zoeter and Heskes (2003); Bakker and Heskes (2007), for instance, introduced dynamic hierarchical models that combine individual time series with group-level dependencies. Hierarchical models have also been applied for forecasting chaotic time series (Matsumoto et al., 1998; Hyndman et al., 2011; Xu al., 2020), or for extracting interpretable summary statistics from time series to capture inter-individual differences (A (Akintayo and Sarkar, 2018; 8; Yingzhen and Mandt, 2018). Much less work exists in the direction of hierarchical DS modeling. LFADS (Pandarinath et al., 2018) and CrEIMBO (Mudrik et al., 2024b) were designed for inferring DS with neuroscience applications in mind, and can integrate observations obtained across different sessions. Roeder et al. (2019), more generally, developed Bayesian framework using variational inference to infer Neural ODE parameters at different hierarchical levels. Yin et al. (2021); Kirchmeyer et al. (2022) used hierarchical models, which decompose dynamics into shared and environment-spedd quality and generalization across different parameter regimes of a DS. Similarly, Bird and Williams (2019) introduced a multi-task DS model where a latent variable encodes task-specific information for sequence generation across different styles/eniiromments. Inubushi and Goto (2020) and Guo et al. (2021) used reservoir computing for transfer learning between chaotic systems, but without explicitly modeling system- sppeciic dynamics. Finally, Desai et al. (2022) proposed a transfer learning approach for PINNs, fine-tuning only the final layer of pre-trained model for ohe-shot learning of ODEs.
components, to enhance forecasting
2

---

Published as a conference paper at ICLR 2025
In all these studies, however, only benchmark systems living in similar dynamical regimes (e.g. all exhibiting periodic orbits) were considered, often only assessing forecasting but not generative performance. This is fundamentally different from the settings with complex, chaotic, topologically diverse dynamics and real-world data we consider here, with a focus on generative and long-term performance across different dynamical regimes.
METHODS
3.1
HIERARCHICAL DSR MODEL
of lengths Tok j 1..... S, such as measurements from related physical systems or from multiple subjects in medical studies. While generally the individual multivariate time series may come from any type of system, in the following we will denote these as 'subjects' as our main application examples will be human data. Our main goal is to infer subject-specific DSR models, parameterized by 0 i.e. generative models of the latent dynamies underlying subject-specific observations. We approximate the dynamics by discrete-time recursive (flow) map of the form
Assume we have observed multiple, multivariate time series (5)
1.. 1I..THo)
(I)
where 8; are possible external inputs (like task stimuli). Observations are related to the dynamical process } via some parameterized observation function
hous (
The differences between subjects captured within low-dimensional parameter space, represented here by learnable subject-specific features (6) E ReNe,, These are mapped onto the parameters 0 (جo) of subject-specifie DSR models through a function parameterized by group-level parameters 2 group common to all subjects:
0 O
Gooouu (1(0)).
The overall approach is illustrated in Fig. 1.
Obserrations
Group-leval parametars (ohered Sponp
Satject-specttic DSR Fyp (4
Reconstructions
subjectioppectte peramatersa 150
with
dlsenatiies
WASSIM FLANNA
i (UANMMN A
subacc
Hest AWA
model
Figure 1: Illustration of the hicrarchization framework.
For our experiments we chose for F (ois) (Eq. 1) an RNN model introduced for DSR in Hess et al. (2023), the shallow PLRNN (shPLRNN).
F 0los (2(2)
((w .0) + n26)) + (1) - 2,2 + W
(4)
with latent states zt M subject-specific parameters o (w()) E MXL w) E RRLM
h(") E RRM, h: E RRL, A E di diag(RA)) and nonlinearity ob.) ReLUU.) max(0, -, which makes the model piecewise-lin -inear and thus mathematically tractable (Eisenmann et al., 2024). We also tested other models for Fo..) including LSTMs (Hochreiter and Schmidhuber, 1997; Vlachas et al., 2018) and vanilla RNNs trained by GTF (Hess et al., 2023), and found that the results reported below for the ShPLRNN stayed essentially the same, see Appx. 1. and Table 3.
3

---

Published as a conference paper at ICLR 2025
Likewise, for mapping the latent states 2, to the observations It, Eq. 2, checked different options, see Appx. A.1.6. Taking simply the identity mapping. a! h (2, (z( 2 (f) as in Hess et al.
(2023), worked about best, presumably because too expressive observation model can remove the burden from the DSR model to account for the actual dynamics (Hess et al., 2023).
Finally, in ablation studies we also explored various ways to parameterize the map Go in Eq. 3, including flexible and expressive functions like multilayer perceptrons (MLPs) and reparame- terizations (see Appx. A.1.7, Table 4), but found that simple linear mappings often gave the best results. Specifically, for the shPLRNN Eq. 4, the subject-level parameters 0 are obtained from 1 N by linearly projecting common group-level parameters f group = {Pw, E (()-LL, Pw. E NN Ph. E R R.NxxMM Pha E RR-a PA E R MN M3 into the subject parameter space according to:
the feature vectors (0) E IR
A = diag(1(), PA); W = mat(l(). Pw,, M,L,, L), / == mat((() Pw; L, L,M). 1 = (1 P,,,", h2 = (1 Pr.,""
where mat(-, m, m, denotes the operation of reshaping a vector into an m x n matrix.
3.2 TRAINING PROCEDURE
All model parameters, i.c. common group-level parameters group (matrices P above) and subject- specific features (():5), are optimized simultaneously (end-to-end) by minimizing the total loss
s T& Σ
2 log EUO1+ + [ao - x -10)) ).
L
(5)
The total loss thus contains additional parameters ( trained jointly with 0 group and (1::)), which subject-specific diagonal scaling matrices introduced deal with data that is different scales across subjects (e.g. for observations of the Lorenz-63 system in the cyclic vs. chaotic regime). For gradient propagation, note that predictions (6) in Eq. 5 are linked via Eq. 2 to the latent model, Eq. 4, whose parameters 0boR are in turn obtained from Ogroup and (6) via Eq. 3, see Fig. 8. For the group-level matrices P. E group Xavier uniform initialization (Glorot and Bengio, 2010) was used and an L2 norm added to the loss, see Appx. A. 1 for more details.
For training, used generalized teacher forcing (GTF) (Hess et al., 2023), SOTA DSR training method specifically designed to address exploding gradients, which are known to occur when training DSR models chaotic time series (Mikhaeil al., 2022). GTF stabilizes training by linearly interpolating between RNN-generated and data-inferred control states in an optimal way (see Appx. A.1). We emphasize that GTF is only used to train the model, while test time the models always run autonomously. RAdam (Liu et al., 2020) was employed as the optimizer, with significantly higher learning rate for the subject-specific feature vectors (10-3) than for the group-level matrices (10-1), which was crucial to prevent numerical instabilities and prioritize incorporation of subject-specific information, see Fig. 19. Across subjects, batching strategy was employed that ensured each gradient update incorporates data from all subjects, see Appx. 1.1.4.
Note that our model, although multi-level, requires only fairly few paper-parameter:: the latent size M (simply taken to be equal to the number of observations in all our tests), hidden size L (for which = 20 x M is a good default), GTF parameter a (which could be determined automatically, see Hess et al. (2023), but here was set to defaults), and the number of features Ntaii see Appx. 1.1.1I.
4 RESULTS
4.1 TRANSFER LEARNING
To illustrate transfer learning, we used two popular DS benchmarks, where we sampled multivariate time series ( with varying ground-truth parameters of the underlying ODE systems: The Lorenz-63 model (Lorenz, 1963) of atmospheric convection (Eq. 15), simulated here with 64 different parameter
4

---

Published as a conference paper at ICLR 2025
combinations ('subjects') with p {21, 81, 18,9.10,11, and [ E (1, and the spatially extended Lorenz-96 model (Lorenz, 1996) with N = 10 dimensions (Eq. 17), for which 20 subjects with F E {10 (j 1) + 1 1=1 were simulated.
The parameter ranges were chosen to cover multiple dynamical regimes, i.c. with fundamentally different attractor topologies (like limit cycles and chaos, Fig. 2). From each parameter setting, sampled only short time series (Tmax = 1000), such that training on individual time series often led suboptimal outcomes. Thus, leveraging group information was crucial for optimal reconstructions. As established in the field of DSR (Wood, 2010; Durstewitz et al., 2023), we evaluated the quality of reconstruction in terms of how well the reconstructed DS captured the invariant long-term temporal and geometric properties of the ground truth DS: We used a state space divergence Dasp to assess geometrical agreement (Koppe al., 2019; Brenner et al., 2022) and the Hellinger distance DH on power spectra (Mikhaeil et al., 2022; Hess et al., 2023) to check temporal agreement between generated and ground truth trajectories (see Appx. A.2).
We compared the performance of our framework to three other recent methods (see Appx. A.4 for details): First, as a baseline we tested an ensemble of individual shPLRNNs, using an otherwise identical training algorithm (a kind of ablation experiment, removing specifically the 'hierarchical component'). Second, we employed LEarning Across Dynamical Systems (LEADS, Yin et al. (2021)), a framework that trains Neural ODEs for generalizing across DS environments by learning shared dynamics model joinlly with environment-specific models. Third, we trained context-informed dynamics adaptation (CoDA, Kirchmeyer et al. (2022)), an extension of LEADS where parameters of the combined and environment-speciine models are drawn from a hypernetwork. As evidenced in Table 1, our hierarchical approach (hier-shPLRNN) considerably outperforms all other setups. In fact, competing methods were often not even able to correctly reproduce the long-term attractor dynamics (Appx. Fig. 17), while our approach successfully recovered different attractor topologies (Figs. & 21). Results stayed the same when vanilla RNNs trained by GTF or LSTMs were swapped in for the shPLRNN in our framework (Appx. 1.1.6 Tab. 3), though LSTMs performed quite poorly overall.
Table 1: Performance of hierarchical and ensemble shPLRNN, CoDA (Kirchmeyer et al., 2022), and LEADS (Yin et al., 2021). Medians (across subjects) + MAD across 10 different training runs.
Dataset
Model
Dey((L)
(J(
# params
# shared params
hier-shPL.R RNNN
0.394 0.014
0.097 + 0.005
6912
63366
Lorenz-63
shPLRNN ensemble
1.82 0.16
0.118 : 0.002
14016
0 96514
CoDA
1.4
0.337 + 0.005
97154
LEADS
9.9
+:
8580
132
Mier-shPLRNN
0.580
0.002
0.0527 + 0.0010
21430
21130
Losenz-96
shPLRNN ensemble
1,291 = 0.023
0.0657 1 0.0012
42630
0
CoDA
2.05 0.07
0.155 0.003
106647
106447
LEADS
3.5
0.54 = 0.06
22575
1075
Scaling and robustness Figs. 12 & 13 show how performance scales with trajectory length Tiux and number of subjects S. While performance of the hier-shPLRRNN is always substantially better than when models
trained individually, this is particularly evident for short time series, Tmax S 500. For Toaxx > 500 performance starts to plateau, but profound gains are still obtained by increasing S up to 16. Surprisingly, training times until given performance level reached actually decrease with the number of subjects S (Fig. 14& & Appx. A.1.9). The reason presumably is that with higher S, the model has access to a larger portion of the underlying system's parameter space, and can leverage this efficiently to speed up training on each single subject. Finally, robustness of discovered solutions across multiple training runs was very high, with feature vectors correlating by r 0.84 0.05 on average across experiments, and as indicated by the > 10-fold smaller MADs in Tab. 1 for the hier-shPLRNN compared to the other approaches (%x 5% of the resp. medians), see also Fig. 11.
4.2 INTERPRETABILITY
Dynamical systems benchmarks We assessed the ability of the hierarchical inference framework to discover interpretable structure from the Lorenz-63 (Eq. 15) and Rössler system (Eq. 16). To this end, we again sampled relatively short time series of length T = 1000 for 10 different values pقه) E {28... 82) for the Lorenz-63, and c0) E {3.8... 4.8) for the Rössler, a range where its dynamics undergoes bifurcation from limit cycle to chaos. To reflect these I-parameter-variation
5

---

Published as a conference paper at ICLR 2025
Tush
900
J003
3500
2000
500
3033
2000
2000
Ground TrA
Conerctes
Gmand Tuth Gererated
1 y 2-2 a
ky
Tiee
Time Strce
80,
10,
8/3
-- 48
Creus
Figure Example reconstructions from hier-shPLRNN trained on short noisy observations (Fmaz = 1000, 5% observation noise) from the Lorenz-63, Rössler and Lorenz-96 systems for different {p. c, F) (settings as in Sect. 4.1 and 4.2). Shown trajectories were freely generated from data-inferred initial state, using only subject-specific feature vectors (6) determine the dynamical regime, and agree in their long-term temporal and geometrical structure with the ground truth (i.c., for times indefinitely beyond the short training sequences).
ground truth settings, we also chose Nieat for the length of the subject-specific parameter vectors. After training, found that the extracted feature values (0) were highly predictive of the ground truth control parameters p(3) and clu), with clearly linear relation (Fig. 3a, see Fig. 23 for the Lorenz-96). The observation that the model automatically inferred such a linear relationship is particularly noteworthy given that the DSR model's piccewise-linear form (Eq. 4) profoundly differs from the polynomial equations defining the ground truth systems (Eqs. 15 and 16). Surprisingly, even when hier-shPL.RNe LRNNS were trained with many more features (Niea 10) than theoretically required, a principal component analysis (PCA) on the feature vectors revealed that a dominant part of the variation was captured by the first PC (> 85% of variance), with negligible contributions beyond the third PC (Fig. 3b, blue curve).' The same results were obtained with a very different DSR model, Markov Neural Operators incl. dissipativiiy prior (Li et al., 2022), embedded into our approach (Fig. 18). In contrast, attempts to extract dimeennniional structure from the parameters of individually trained models were unsuccessful, with variation distributed across many PCs (Fig. 3b, gray curve).
a 80
4.6
0.8
Individual mode's
Predicted
60
Predicted
4.4
Explained Varience
0.5
Feature vectors
4.2
0.4
40
0.2
3.8
0.0
30
40
50
60
70
80
3.8
4.0
4.2
4.4
4.6
Ground Truth
Ground Truth
Principol Components
Figure 3: Analysis of one-dimensional features for hierarchical models trained observations from the Lorenz-63 (left) and Rössler (right) system for different values p and c. b: Explained variance ratio of the feature space PCs for a hier-shPLRNN trained on the Lorenz-63 with N = 10 vs. ratio for PCA directly on the parameters of individually trained models.
Multidimennsooall parameter spaces We next examined a scenario where all three of the ground- truth parameters a, P and 3 of the Lorenz-63 (Eq. 15) were jointly varied across grid of 4x 64 subjects, as in Sect. 4.1, covering both periodic and chaotic regimes. While naturally, in this case, scalar subject-specific feature did not capture the full variation in the dataset, we found that good
IIteresstingyy, observed that additional PCs reflected higher powers of the control parameters, i.c. PC2 related to p² and PC3 p² for the Lorenz663.

---

Published as a conference paper at ICLR 2025
reconstructions could already be achieved using Niea 3. while using Niaa 6 features led to optimal performance. The extracted features remained highly interpretable, as illustrated in Fig. 4: strongly aligned with the first and with the second PC of the subject feature space, while variation in appeared nested within steps of /3 across PC2. The eight subjects on the left which do not fall into the 4 x grid with the others, correspond to parameter combinations that put the system into a non-chaotic, cyclic regime. These subjects do not only form distinct group in the first two PCs, but also have significant non-zero third and fourth PCs (see Fig. 22). This observation helps explain why the algorithm benefits from additional features (Nieat > 3), as these allow to represent different dynamical regimes as distinct regions feature space, thereby further supporting its interpretability.
Fig. 28 further demonstrates that this semantic structuring of the feature spaces is observed even when the hier-shPL.RNN trained simultaneously on different dynamical regimes from three completely different DS, the Lorenz, the Rössler and the Chua system. The different DS are widely separated in feature space, and are further segregated according to dynamical regime within given DS
10
11
21
31
81
11:
0.6
0.4
0.4
0.2
0.2
0.2
-0.2
-0.2
-0.2
-0.4
-0.4
-0.4
-0.3 PC1
-0.5
Figure 4: PCA projection of the 6d feature space for hier-shPLRNN trained on the Lorenz-63 with variation across all 3 ground truth parameters (each dot represents one parameter combination or 'subject'). From left right, color-coding corresponds to parameters o, p and (3, respectively. The leamed feature space is highly structured and clearly distinguishes different dynamical regimes.
Capturing non-stationarity with time-dependent features One interesting use case could be the different systems are functionally related, e.g., reflect different temporal snapshots of non- autonomously evolving DS with time-dependent control parameter(s). In Fig. 16 we simulated such a scenario for the Lorenz-63 with p(t) E [28, 68) slowly drifting across time. Co-training an additional function ((") M i) resulted in an explicity time-dependent DSR model which almost perfectly captured the temporal drift in p(t) (linear R² = 0.995), confirming that our approach could in principle easily be amended to handle such situations.
Simulated arterial pulse waves We then tested our approach on a much higher-dimennsionll (N 52) example system, closer relevant real-world scenarios, a biophysical model of arterial pulse waves (Charlton et al., 2019). The dataset consists of 4,374 simulated healthy adults aged 25-75 years, with four modalities (pressure, flow velocity, luminal area, and photoplethysmogram across 13 body sites (see Appx. A.3 for details). Additionally, the dataset includes 32 haemodynamic parameters, such as age, heart rate, arterial size and pulse pressure amplification (Table 6), which specified the biophysical model simulations.
After training, we found that the hier-shPLRNN became an almost perfect emulator of the data, with freely generated trajectories" closely matching the ground truth pulse waves (Fig. 5a and Appx. Fig. 24 for further examples). This is particularly impressive given that the number of parameters of the hier-shPLRRN constituted less than 1% of the total amount of training data points, and comparatively low-dimensional (in relation to the number of biophysical parameters) feature vector (Nrea 12) was sufficient capture individual differences between subjects. The model's ability to achieve these reconstructions with so few parameters indicates that it extracted meaningful structure from the data, rather than merely memorizing Accordingly, the extracted feature vectors could predict the haemodynamic parameters via linear regression with high and in all cases statistically
By freely generated mean that these are not merely forward predictions, but new trajectories drawn from the generative DSR model using only data-inferred initial condition.
7

---

Published as a conference paper at ICLR 2025
0.25
Tampprrl Anay Astertal Foa Walecy Gound Trah
SE
Time [s]
frsty Gonersiod tffourctod sthurnti
0.13
0.10
dng 778
0.09
MBP 9
0.0
0.1
0.2
0.5
0.4
2.5
0.5
0.7
KPV
10
00
02
0.1
0.6
0.
R Valuee
044
Prodeeding
800
Howercyramm
275
Delkired 0.1
0.2
paloo vowa miodh (99)
FC4
Ground
Ground
Prinoipal
Figure 5: Ground truth and simulated (freely generated) arterial flow velocity waves for 10 representative subjects (selected by k-medoids, color-coded). b: R² scores for the 32 ground truth haemodynamic parameters predicted from the learned feature vectors via linear regression. Ground truth and predicted values for the left ventricular ejection time (LVET) and pulse wave velocity (PWV). d: Variance explained by the first 5 principal components of the learned feature space (Nreat = 12) and the 32d space spanned by the haemodynamic parameters.
significant (F-tests, < 0.05) accuracy (mean R² 0.83, Fig. 5b). Two parameters with particularly strong relationship (A² > 0.97) are in Fig. 5c. Moreover, the explained variances the first five principal components of the feature space agreed well with that across the haemodynamic (biophysical) parameter space (Fig. 5d), indicating that the learned feature space accurately captured the pattern of variation in biophysical dynamics. Again, these results are highly non-trivial as the model was trained only on the arterial pulse waves without any knowledge of the biophysical ground truth parameters.
4.3 WEW-SHOT-LEAARINS
To test 'few-shot learning', by which we mean here the ability to infer new models from only small amounts of data, first trained hier-shPLRNN on S
subjects simulated using the Lorenz-63 with Puain = {28, 33.8, 39.6, 43.3, 56.9, 62.7, 68 4, 74.2, 80). We then generated new, short sequences lees with Tmax 100, using values of Piest randomly sampled from the same interval [28, 80] that also contained the training data, and fine-tuned only scalar feature ( this test set. The model closely approximated (interpolated) the true Piest values from these single (5) short sequences (Fig. 6a), and was even able extrapolate to new Piest outside the training range. Fig. 6b exemplifies how estimates become increasingly robust for longer sequences. This result with 1000 time steps (see Table & Fig. 12). Even for sequences as short as T 7ma = 100, the loss curves were often uni-modal and smooth around the true parameter values (Fig. 6c), observed previously for training with sparse or generalized teacher forcing (Hess et al., 2023; Brenner et al., 2024). This allowed gradient descent converge rapidly, within seconds Core i7- 1800H. It also enabled efficient use of 2nd -oderr Hessian-basee based methods, which converged within just second. Hence, even with minimal data fast and reliable estimation of previously unseen dynamics is feasible, extending into regimes beyond the training domain.
(i)
noteworthy training an individual shPLRNN for the Lorenz-63 is challenging
single 1 Gen 2.30 GHz Intel
Further, albeit small, imprwements in accuracy could be achieved by nonlinear regression via random forests (mean R² (1.87).
8

---

Published as a conference paper at ICLR 2025
a
125
70
Mean predicted p
10
100
Prodlcsed
10
50
10
P
30
80 200
140
Thiiring Sheps
Predicted
Figure 6: a: Predictions (by linear regression as above) for 25 Piest values randomly drawn from the training range (green vertical lines) and several Piest values extrapolated beyond the training range, which only the scalar subject feature (Nieat 1) fine-tuned single short test sequences mT = 100) (see Appx. A.1.10 on how estimates of standard deviation were obtained). b: Predictions of flest as a function of training sequence length. e: GTF loss for short training sequence (Tmux = 100) across different values of the scalar subject feature mapped onto via linear regression. Note the uni-modal loss centered on the true ptest even for this short test set.
())
4.4 EXPERIMENTS WITH HUMAN EMPIRICAL DATA
Unsupervised extraction of DS feature spaces from elinical electroencephalogram (EEG) data In the previous sections, we directly linked extracted feature vectors to known ground truth parameters. However, in most real-world scenarios, ground truth parameters are unavailable and we only have indirect information like class labels, making feature interpretation more challenging. To assess the performance of our hierarchical framework in such settings, we trained it on human EEG data from subjects who were either healthy experiencing epileptic seizures (Zhang et al., 2022; Andrzejak et al., 2001), as before in a completely unsupervised fashion (i.e., ignoring known class labels). Using Gaussian Mixture Models (GMMs) for clustering in the extracted DS feature space, able to recover the true class labels with an accuracy of 92.6 + 1.2% across ten runs, substantially outperforming several leading approaches for unsupervised feature extraction from time series, such as tsfresh, (Christ et al., 2018), Catch-22, (Lubba et al., 2019), ROCKET (Dempster et al., 2020), MiniRocket (Dempster et al., 2021), and attention- based convolutional autoencoders (Wang et al., 2023) (see Table 7). Fig. 7 shows that the two classes form distinct clusters clearly separated along the first PC of the training data, similar to the clustering different dynamical regimes in Fig. 4, but unlike the other unsupervised techniques where classes were much less distinct. These findings confirm that clustering according to DS properties may profoundly improve on 'classical' time series feature extraction, as conjectured. Fig. 7 further illustrates why miscclassisications happened.
Transfer learning and classification on human fMRI data We further tested our hierarchical DSR approach on set of short fMRT time series from 2 (hemispheres) 100 distinct brain regions from 26 human subjects participating in a cognitive neuroscience experiment (Koppe et al. (2014), see Appx. A.3), of which 10 with minimal irregularities or artifacts were selected. Based on PCA (Fig. 26c), we determined that feature vector dimension of Nica 10 was necessary to adequately capture the unique characteristics of the individual time series (Fig. 25a). For left-out subject, reconstruction of the overall dynamics was feasible even when only fine-tuning the low-dimensional feature vector on unseen data (Fig. 25b). Since for this dataset we did not have any ground truth labels or classes, we sought to assess the robustness of feature extraction by computing the cosine similarity matrix across twenty training runs. Strong correlations (r zs 0.85 = 0.05) indicated that feature vectors were indeed robustly inferred. Clustering the features via GMMs revealed distinct groups that aligned with visual differences observed in the fMRI time series (Fig. 26).
Finally, Fig. 27 illustrates that our hierarchical DSR approach also works for more 'classical' time series data (electricity consumption). beyond the domain of scientific applications.
5 CONCLUSIONS
While DSR is currently a burgeoning field, the question of how to best integrate dynamically diverse systems into common structure, and how to harvest this for transfer learning, generalization, and classification, only recently gained traction (Yin et al., 2021; Kirchmeyer et al., 2022; Göring et al., 2024). Unlike TS models, where the interest lies mainly in forecasting TS classification/ regression,

---

Published as a conference paper at ICLR 2025
a
tdhesh,
MisRocket scunsgeeass
PC1
PCE
PC
Eplletic fernect Hos lonnact
Extiptic cresc
Healle sessisss Bird)
Calth 22, 00080ry-57712..5
AotrCor scour
MAMMM
www mmm
Mealhhy
www.
Epdeptic intectoodiee
20
15
Ground Truth
Freelly generated
Figure 7: a: First two principal components of the embedding spaces of the hier-shPLRNN and other unsupervised time series feature extraction approaches from Table 7. Ground truth labels are indicated by color (blue: epileptic, red: healthy) and predicted labels are indicated by markers (circle: predicted epileptic, cross: predicted healthy). The DS feature space of the hier-shPLRNN exhibits the clearest separation of epileptic and healthy subjects (as confirmed quantitatively). b: Left: Ground truth data for two correctly and one incorrectly classified subjects, colored by ground truth labels. trajectories, colored by predicted labels. Note that the incorrectly classified subject is indeed dynamically much more similar to the 'healthy' than the 'epileptic' EEG.
Right:
in DSR we aim for generative models that capture invariants of the underlying system's state space. Using GTF (Hess et al., 2023) for training, we demonstrated that hierarchically structured models significantly improve DSR quality compared to non- non-hierarchical methods. This is strong indication that our method was able to utilize information from all DS observed, even if located in different dynamical regimes, or if described by different systems of equations, to boost performance on each individual system. This was confirmed in transfer learning experiments, where models trained on multiple subjects generalized effectively to new conditions with minimal additional data, and did so much more convincingly than various competing methods. Identifying dynamical regimes, instead of just utilizing TS waveform features, is also likely to be a more principled approach to TS foundation models, since it is the underlying dynamics which determine system's temporal properties across contexts. This idea is supported by the strong classification results on human EEG data, where subject groups were much more clearly segregated in the DS feature space than in spaces constructed by "conventional" TS models. These DS feature spaces further turned out to be surprisingly interpretable. Extracting these dynamically most crucial features and grouping subjects along these dimensions may be of considerable interest in areas like medicine or neuroscience, and is a new application that has not been considered in the DSR field so far.
Limitations The highly interpretable nature of the subject-level feature spaces is surprising. It appears that our method automatically infers critical control parameters of system that best dif- ferentiate between dynamical regimes (by controlling the underlying bifurcation structure), but it is currently unclear what the mathematical basis for this might be. Another open issue is how DS could best be integrated into the same model that are related but live on widely different time scales, e.g. brain signals over minutes vs. behavioral monitoring across weeks. While Fig. 28 and the empirical examples confirm our approach provides a means to examine how diverse systems are dynamically related, this currently is limited to DS evolving on similar time scales. In Fig. 16 we illustrated how the hier-shPLRNN could be expanded to capture non-stationarity and drivers in non-autonomous DS in an interpretable way: yet, for specific physical or biological processes this may need to be hooked up with more domain knowledge to establish a mechanistic link to the underlying physics/ biology. A useful extension thus could be the incorporation of prior physical knowledge in SINDy PINNs. While the latter is no principle obstacle, as shown here for Markov Neural Operators, the tricky part may be to formulate priors that work across all observed systems and domains.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
The code for this paper is publicly available at https///shhhbbooooonrrrneiiiittt
ACKNOWLEDGEMENTS
This work was funded by the European Union's Horizon 2020 programme under grant agreement 945263 (IMMERSE), by the German Ministry for Education & Research (BMBF) within the FE- DORA (01EQ2403F) consortium, by the Federal Ministry of Science, Education, and Culture (MWK) of the state of Baden- worden-Wirttebberg within the AI Health Innovation Cluster Initiative and living lab (grant number 2/3//2223-7/// by the German Research Foundation (DFG) within the collaborative research center TRR-265 (project A06 & B08) and by the Hector-II foundation.
REFERENCES
Adedotun Akintayo and Soumik Sarkar. Hierarchical symbolic dynamic filtering of streaming non-stationary time series data. Signal Processing, 151:76 88, October 2018. ISSN 0165-1684. doi: 2011
URL https: // http://wwieneee.eeeeee scwencedi rect, com science/
article/pii/sol 651 6841 00866881115158 8301506.
Ralph G Andrzejak, Klaus Lehnertz, Florian Mormann, Christoph Ricke, Peter David, and Christian E Elger. Indications of nonlinear deterministic and finite-dimensional structures time series of brain electrical activity: Dependence on recording region and brain state. Physical Review E, 64 (6):061907, 2001.
Omri Azencot, N. Benjamin Erichson, Vanessa Lin, and Michael W. Mahoney. Forecasting Se- quential Data using Consistent Koopman Autoencoders. In Proceedings of the 37th International Conference on Machine Learning. 2020. URL http: orgg abs 2003. 02236.
Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling, April 2018. URL http:// /ttp:://axxvvvvoo org/ abs/ 1803. 01200.111111
Stt.:::803301211
Bart Bakker and Tom Heskes. Learning and approximate inference in dynamic hierarchical models. Computational Statistics & Data Analysis, 52(2): 839, October 2007. ISSN 0167-9473. doi:
URL https: www. sciencedi rect ctt.ttcoosooeooooooe/oooeeeee.co ctom/scceee science/
Mark Berliner. Hierarchical Bayesian Time Series Models. In Kenneth M. Hanson and Richard N. Silver, editors, Maximum Entropy and Bayesian Methods, pages 15-22, Dordrecht, 1996. Springer Netherlands. ISBN 111-994--11-5430--1 doi: 10. 00.10007778--4-011--55 --11---------
Alex Bird and Christopher KI Williams. Customizing sequence generation with multi-task dynamical systems. arXiv preprint arXiv:1910.05026 2019.
Manuel Brenner, Florian Hess, Jonas M. Mikhaeil, Leonard F. Bereska, Zahra Monfared, Po- Chen Kuo, and Daniel Durstewitz. Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems. In Proceedings of the 39th International Conference on Machine Learning, pages
PMLR. June 2022. URL https::/pprcceeee
mlr. mlr.press/vv66
brenner html. ISSN: 2640-3498.
Manuel Brenner, Florian Hess, Georgia Koppe, and Daniel Durstewitz. Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics. In Proceedings of the 41st In- ternational Conference on Machine Learning, pages 4482-4516. PMLR, July 2024. URL ISSN: 2640-3498.
https: httpsss/ppccceeddnee
Steven L Brunton and Nathan Kutz. Data-driven science and engineering: Machine learning,
dynamical systems, and control. Cambridge University Press, 2019.
Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Discovering governing equations from data by sparse identification nonlinear dynamical systems. Proceedings of the National Academy of Sciences USA, 113(15):3332-397) 2016. ISSN 0027-8424. doi: 10. 10.1073/pnas... 15173341
11

---

Published as a conference paper at ICLR 2025
Steven L. Brunton, Marko Budišić, Eurika Kaiser, and J. Nathan Kutz. Modern Koopman Theory for Dynamical Systems, October 2021. urXiv:2102.114 12086 [cs, eess, math].
Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza, Max Mergenthaler-Came and Artur Dubrawski. N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting, November 2022. URL http://tt//// //axivv.aiivvvv/aa wwarxiivorg//.eeoo wwt//bs/////ooooooooo...oo...err 2201 12886. arXiv:2201.122 12886.
Peter H. Charlton, Jorge Mariscal Harana, Samuel Vennin, Ye Li, Phil Chowienczyk, and Jordi Alastruey. Modeling arterial pulse waves in healthy aging: database for in silico eval- uation of hemodynamics and pulse wave indexes. American Journal of Physiology- and Circulatory Physiology, 317(5) H1085, November 2019. ISSN 0363-6135. doi: 10.1 11152//apheartt000112220000 URL https: / journals parssohysooooo ogy org /doi/// /fll//ful////// 10. 10.1152/ a 00218 8 2019. Publisher: American Physiological Society.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Maximilian Christ, Nils Braun, Julius Neuffer, and Andreas W. Kempa-Liehr. Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package). Neurocomputing. 307:72-77, September 2018. ISSN 0925-2312. doi: www.://....cmmm2011333333 067.. URL https: http:://c/eeee/00222221111100000 //www. www.wscienceei.eom rectt com www.sscieccceeeee /ttp:///ceceeeeeeeaaaaaaaaaaa///
L. Chua, M. Komuro, and T. Matsumoto. The double scroll family. IEEE Transactions on Circuits and Systems, 33(11):1072-1118, November 1986. ISSN 1558-1276. doi: 10.1 1ww.1199//CSS. 1986. 1085869. URL / /eeexplore Leee.org http://ccmmeetnteeeeee htt.o//re.eee.oococouuee/ee///ee Transactions on Circuits and Systems.
Conference Name: IEEE
Angus Dempster, François Petitjean, and Geoffrey Webb. ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Dis- covery, 145 September 2020. ISSN 1573-756X. doi: 10.1007/s10618 1000000000000000000----2
URL hwwphhtp::/tm /ww.i/ooiiiio///// http://///...1111111111000//01.0 10..0..11000111100 0tt1://111611000000000000 0200000000000022222-
Angus Dempster, Daniel F. Schmidt, and Geoffrey I. Webb. MiniRocket: A Very Fast (Almost) Deterministic Transform for Time Series Classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21, pages 248-257, New York, NY, USA, August 2021. Association for Computing Machinery. ISBN 978-1-45003-332-55 doi: 10.1 10.1144534445548.447731. URL https: http:///tooogg/////////o O. 11445/3445545 3467231..
Shaan Desai, Marios Mattheakis, Hayden Joy, Pavlos Protopapas, and Stephen Roberts. One-Shot Transfer Learning of Physics-Informed Neural Networks, July 2022. arXiv:21 10.11286 [physics].
Daniel Durstewitz, Georgia Koppe, and Max Ingo Thurm. Reconstructing computational system dynamics from neural data with recurrent neural networks. Nature Reviews. Neuroscience, 24(11): 693-710, November 2023. ISSN 1471-0044 doi: 10. 10.10088/41533--020000000-7777..
Lukas Eisenmann. Zahra Monfared, Niclas Göring, and Daniel Durstewitz. Bifurcations and loss jumps in RNN training. Advances in Neural Information Processing Systems, 36, 2024.
Janik Fechtelpeter, Christian Rauschenberg, Hamidreza Jamalabadi, Benjamin Boecking, Therese van Amelsvoort, Ulrich Reininghaus, Daniel Durstewitz, and Georgia Koppe. A control theoretic approach to evaluate and inform ecological momentary interventions, February 2024. Publisher: OSF.
Stathi Fotiadis, Mario Lino Valencia, Shunlong Hu, Stef Garasto, Chris D. Cantwell, and Anil Anthony Bharath. Disentangled Generative Models for Robust Prediction of System Dynamics. In Proceed- ings of the 40th International Conference on Muchine Learning, pages 10222-10248. PMLR, July 2023. URL https: htt..tt//////rroeeeeeeem pww.eedingggeeeeee mlr press fot s23a html. ISSN: 2640-3498.
K. Gajamannage, D. I. Jayathilake, Y. Park, and E. M. Bollt. Recurrent neural networks for dynamical systems: Applications to ordinary differential equations, collective motion, and hydrological modeling. Chaos, 33(1):013109, January 2023, ISSN 1054-1500. doi: 00.800/500008748. URL
https: //ww:/ ncbi. m1m. n htthho//mme//rtticlee///eeeeee http:///rttticeee///2222333/
12

---

Published as a conference paper at ICLR 2025
Andrew Gelman and Jennifer Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models, December 2006. ISBN: 9780511790942 Publisher: Cambridge University Press.
William Gilpin. Generative learning for nonlinear dynamics. Nature Reviews Physics, 6(3):194 206, March 2024. ISSN 2522-5820. doi: 10. 10101///42234404-00000553333 URL https: / /www. nature. www./c/atatoo.eeo....oe es 22244 -24-000---000 00688-2.8.2288
Publisher: Nature Publishing Group.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks, In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 249-256. JMLR Workshop and Conference Proceedings, March 2010. ISSN: 1938-7228.
Harvey Goldstein. Multilevel models in education and social research. Multilevel models in education and social research. Oxford University Press, New York, NY, US, 1987. ISBN 978-0-85264-288-7 978-0-19-5206188 Pages: viii, 98.
Yali Guo, Han Zhang, Liang Wang, Huawei Fan, Jinghua Xiao, and Xingang Wang. Transfer learning of chaotic systems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(1):01 104, January 2021. ISSN 1054-1500. doi: 10. 11161655000337777 URL https://doi. org/10.
1063/5 0033870.
Niclas Alexander Göring, Florian Hess, Manuel Brenner, Zahra Monfared, and Daniel Durstewitz. Out-of-Domain Generalization in Dynamical Systems Reconstruction. In Proceedings of the 41st International Conference on Machine Learning, pages 16071-16114 14. PMLR, July 2024. URL https: http:::/pprceeeiie proceedings nigs ml press v255//23///////// httl///ornngg//0/ html. ISSN: 2640-3498.
Julien Herzen, Francesco LArssig, Samuele Giuliano Piazzetta, Thomas Neuer, LÅ@o Tafti, Guil- laume Raille, Tomas Van Pottelbergh, Marek Pasicka. Andrzej Skrodzki, Nicolas Huguenin, Maxime Dumonal, Jan KoAscisz, Dennis Bader, FrA@dλOrick Gusset, Mounir Benheddi, Camila Williamson, Michal Kosinski, Matej Petrik, and GaAxd Grosch. Darts: User-friendly modern machine learning for time series. Journal of Machine Learning Research, 23(124):1-6, 2022. URL http:// jmlr F. org/ papers v23
2223/221 22/3/112222 11-11777 html.
Florian Hess, Zahra Monfared, Manuel Brenner, and Daniel Durstewitz. Generalized Teacher Forcing for Learning Chaotic Dynamics. In Proceedings of the 40th International Conference on Machine Learning, pages 13017-13049. PMLR, July 2023. URL https: htttp:://proceedings..
mlr. hlr.presss/v222/1e https///w22hhhesss2em hess22a
html. ISSN: 2640-3498.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neurul Comput., 9())
1735-1780, nov 1997. ISSN 0899-76677 doi: (0.1162/nco.1997. 1997.9.8.1735.
Rob J. Hyndman, Roman A. Ahmed, George Athanasopoulos, and Han Lin Shang. Optimal com- bination forecasts for hierarchical time series. Computational Statisties & Data Analysis, 55(9): 2579-2589, September 2011. ISSN 0167-9473, doi: 10. 10.101166.cccaa.2111...000 URL https: www. . sci enceii rect. cwm/ htt//sciencee htt.://eeeeeeerraeeee///////////
Masanobu Inubushi and Susumu Goto. Transfer learning for nonlinear dynamics and its application to fluid turbulence. Physical Review E, 102(4):043301, October 2020. doi: PO...... 102.043301. URL htttt::: link. aps.aapp.... ht0t:///ab.......//////eeeeeee 102. 043301. Publisher: American Physical Society.
Ruoxi Jiang, Peter Y. Lu, Elena Orlova, and Rebecca Willett. Training neural op- erators preserve invariant measures of chaotic attractors.
Advances
Neu-
ral Information Processing Systems, 36:27645-27669, ptttps://proceednngt neurips 5 (ateee7ee599adl11aa (eee666882288aa55440 Abstract
December 2023.
URL
wwtpccppppppppccceee www..pp..ccpparrrrr.... htt.ea/aaeeeasttttte---ooeeeeeee Conference html.
E. Kaiser, N. Kutz, and S. L. Brunton. Sparse identification of nonlinear dynamics for model predictive control in the low-data limit. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 474(2219):2018033 November 2018, doi: 10. 1...9////p..2118.3333.... Publisher: Royal Society.
13

---

Published as a conference paper at ICLR 2025
Matthieu Kirchmeyer. Yuan Yin, Jérémie Donå, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari. Generalizing to New Physical Systems via Iontext-Informed Dynamics Model, June 2022. URL http::taaxii..o//etee/// org/ abs / 220//222.2200 01889. arXiv:2202.01 [cs, stat].
Joon-Hyuk Ko, Hankyul Koh, Nojun Park, and Wonho Jhe. Homotopy-based training of NeuralODEs for accurate dynamics discovery, May 2023. URL arXiv:2210.01424 [physics].
Georgia Koppe, Harald Gruppe, Gebhard Sammer, Bernd Gallhofer, Peter Kirsch, and Stefanie Lis. Temporal unpredictability of a stimulus sequence affects brain activation differently depending on cognitive task demands. Neurolmage, 101:236 2444, 2014. ISSN 1095-9572. doi: 10.1016/j.
neuroimage.2014 2014....11.....000
Georgia Koppe. Hazem Toutounji, Peter Kirsch, Stefanie Lis. and Daniel Durstewitz. Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRL. PLOOS Computational Biology, 15(8)=e0007633, 2019. ISSN 1553-7358. doi: 10. 10.1371/journal.pe 1007263.
Nan M. Laird and James H. Ware. Random-Fiffects Models for Longitudinal Data. Biometrics, 38(4): 963-974, 1982. ISSN 0006-341X. doi: 10.2307/2529876. URL https / ///:://ww// /wwp/w/..j www.j.stor.oog. (5able/252981 Publisher: [Wiley, International Biometric Society].
Zongyi Li, Miguel Liu-Schiaffini, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Learning chaotic dynamics in dissipative systems. Advances in Neural Information Processing Systems, 35:16768-16781, 2022.
Scott Linderman, Matthew Johnson, Andrew Miller, Ryan Adams, David Blei, and Liam Paninski. Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems. In Proceed- ings of the 20th International Conference on Artificial Intelligence and Statistics, pages 914-922. PMLR, April 2017. URL https: http:///poccencngeeeeeee ml www.spresssv....... http///4/1///eee Linde rman17a.... html. ISSN: 2640-3498.
Liyuan Liu, Haoming Jiang, Pengeheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. In Proceedings of the Eighth International Conference Learning Representations (ICLR 2020), April 2020.
Edward N Lorenz. Deterministic nonperiodic flow. Journal of atmospheric sciences, 20(2):130-14) 1963.
Edward N Lorenz. Predictability: A problem partly solved. In Proc. Seminar on predictability. volume 1, 1996.
Carl H. Lubba, Sarab S. Sethi, Philip Knaute, Simon R. Schultz, Ben D. Fulcher, and Nick S. Jones. catch22: CAnonical Time-series CHaracteristics. Data Mining and Knowledge Discovery, 33 (6):1821-1852, November 2019. ISSN 1573-756X. doi: 10.1007/1100188-199-04777 URL https: http:://doooooog///// 10100/00001110
009900047-xx
T. Matsumoto, H. Hamagishi, J. Sugi, and M. Saito. From data-to dynamics: predicting chaotic time series by hierarchical Bayesian neural nets. In 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH136227), volume 3. pages 2535-2540, Anchorage, AK, USA, 1998. IEEE. ISBN 978-0- 7803-4859-2. doi: 10. (occuentt687261...
URL http: http::/ieeexploree weee..reeerr... .rg/
Charles E. McCulloch, Shayle R. Searle, and John M. Neuhaus. Generalized, Linear, and Mixed Models. John Wiley & Sons, September 2011. ISBN 978-1-118-20996-7. Google-Books-ID: kyvgyK_sBIkC.
Andreas Meyer-Lindenberg The non-ergodic nature of mental health and psychiatric disorders: implications for biomarker and diagnostic research. World Psychiatry, 22(2):272-25 June 2023.
ISSN 1723-8617. doi: 10. 10.1002/wps.21086 URL https: /ww. ncbi nlm, nih. nih.gov/
pmc/articles//wae 00MM119911199/
14

---

Published as a conference paper at ICLR 2025
Jonas Mikhaeil, Zahra Monfared, and Daniel Durstewitz. On the difficulty of learning chaotic dynam- ics with RNNs. Advances in Neural Information Processing Systems, 35:11297-11312, Decem- ber 2022. URL
neurips cc/ paperr wwwes//pppeeee////////// htt.rasssstttooooooeaoaaaaa-aea- Abstractt-onfereeeee html.
S. Chandra Mouli, Muhammad Ashraful Alam, and Bruno Ribeiro. MetaPhysiCa: 0OD Robustness in Physics-informed Machine Learning, March 2023. URL htlLtttt////xi..rrrrrgg 03181. arXiv:2303.0318 [cs, stat].
N. Mudrik, Y. Chen, E. Yezerets, C. Rozell, and A. S. Charles. Decomposed linear dynamical systems (dIds) for learning the latent components of neural dynamies. Journal of Machine Learning Research, 25(59):1-44 2024a.
N. Mudrik, R. Ly, O. Ruebel, and A. S. Charles. Creimbo: Cross ensemble interactions in multi-view brain observations. arXiv preprint, 2024b.
Ilan Naiman and Omri Azencot. A Koopman Approach to Understanding Sequence Neural Models. arXiv:2/0.07824 fes, math], October 2021. URL http: / arxiv, org
07824.
Adrian M Owen, Kathryn M McMillan, Angela R Laird, and Ed Bullmore. N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human brain mapping, 25(1):46-59, 2005.
Sinno Jialin Pan and Qiang Yang. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345-133 October 2010. ISSN 1558-2191. doi: 10. 10.0T0/TK 2009.191 URL https: / eeexpl ore. feee orgg httg//docmeettt5555 Name: IEEE Transactions on Knowledge and Data Engineering.
Conference
Chethan Pandarinath, Daniel J. O'Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D. Stavisky, Jonathan C. Kao, Eric M. Trautmann, Matthew Kaufman, Stephen Ryu, Leigh R. Hochberg, Jaimie M. Henderson, Krishna V. Shenoy, L. F. Abbott, and David Sussillo. Inferring single-trial neural population dynamics using sequential nuto-encoders. Nature Methods, 15(10):805-815 doi: 10. 10.1008//41592.00 com/ htt//attclees.. s 592441922-500 01 8-00000----9
2018. ISSN 1 1548-7105.
URL https: /ww. nature.
Jaideep Pathak, Zhixin Lu, Brian R. Hunt, Michelle Girvan, and Edward Ott. Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data. Chaos: An Interdisciplinary Journal of Nonlinear Science, 12(12):22110,, December 2017. ISSN 1054- 1500, 1089-7682. doi: 02.1663/155010300 URL UURLtttp://accvv.or////// http://aaa//111003/113 arXiv: 1710.07313.
Jason A. Platt, Stephen G. Penny. Timothy A. Smith, Tse-Chun Chen, and Henry D. I. Abarbanel. A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal 08910. arXiv:2201.08910
Dynamics, Januaryy 2022. BtLptt//p//attt http//aaxi......aaaaa/a/ [cs].
Jason A Platt, Stephen G Penny. Timothy A Smith, Tse-Chun Chen, and Henry DI Abarbanel. Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers. Chaos: An Interdisciplinary Journal of Nonlinear Science, 33(10), 2023.
M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equa- tions. Journal of Computational Physics, 378:686-707, February 2019. ISSN 00219991. doi: jc. 10.004.. URL https: /ttik/iiiiinnhhh el elsevier wwwm/reerieeee. p11//00211991118000
Geoffrey Roeder, Paul K. Grant, Andrew Phillips, Neil Dalchau, and Edward Meeds. Efficient Amor- tised Bayesian Inference for Hicrarchical and Nonlinear Dynamical Systems. arXiv:1905.1200 [cs, stat), October 2019. URL http:// /ttp///rxvvv/////////////////.aa wwtpxiivorggaaae org/abs / 1905....... 12090. arXiv: 1909.
O. E. Rössler. An equation for continuous chaos. Physics Letters A, 57(5):997-398, 1976. ISSN 0375-9601. doi: 000101600375-96011666999 http:///innne/rrrdaaaeee////////
URL https: www. s sciencedirect.
15

---

Published as a conference paper at ICLR 2025
Yair Schiff, Zhong Yi Wan, Jeffrey B. Parker, Stephan Hoyer, Volodymyr Kuleshov, Fei Sha, and Leonardo Zepeda-Núñez. DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems. In Proceedings of the 41st International Conference Machine Learning. pages 43649-44 43644 PMIR, July 2024. URL htw.t:t////// httpp/ooooodggg//o ngs .ml www..m.ws.......mmmmmmmmm......e poress/v233
schi FF24b. html. ISSN: 2640-3498.
Abhinav Subramanian and Sankaran Mahadevan. Probabilistic physics-informed machine learning for dynamic systems. Reliability Engineering & System Safety, 230: February 2023. ISSN
0951-8320, doi: 10.1016/ www.:/o.....ss..22......... 108899. URL https:
http://wwwwwccieeeiieeeeee. com/
Artur Trindade. Electricity load diagrams (2011-2014).
UCI Machine Learning
Repository, 2015.
URL ptttps///rrchiie -ics uui. edu/datasst/3211
Accessed: 2024-11-15.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017. URL http: arxiv. org/ abs/
1706. 03762.
Pantelis R. Vlachas, Wonmin Byeon, Zhong Y. Wan, Themistoklis P. Sapsis, and Petros Koumout- sakos. Data-driven forecasting of high-dimensional chaotic systems with long short-term mem- ory networks. Proc. R. Soc. A., 474(2213):2010844,, 2018. ISSN 1364-5021, 1471-2946. org. /oi/
doi: 10. (0090///aa.201..084444 URL https: /ttpo/tttto//oooooosooteeeeteee
1.98 rspa. 00017084400
Qixin Wang, Kun Qin, Binbin Lu, Huabo Sun, and Ping Shu. Time-feature attention-based convo- lutional auto-encoder for flight feature extraction. Scientific Reports, 13(1):14175, August 2023. doi: 10. 1038//011/99.
ISSN 2045-2322.
000-44421----2222299990 023-41295-y. URL https www. wwwure..t..... com/ 41295- Publisher: Nature Publishing Group.
articles /aceees//419955
Simon N. Wood. Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466
(7310):1122-1104, August 2010. ISSN 1476-4687. doi: 10. 1038///aure09911, URL https:
www. www.untuuee.... com http:/rsticeeeeeee Group.
hwt.c///.....ccle/aaate00030000 http://nuuuuuuuaaeeeeeee Number: 7310 Publisher: Nature Publishing
Meiling Xu, Min Han, C. L. Philip Chen, and Tie Qiu. Recurrent Broad Learning Systems for Time Series Prediction. IEEE Transactions on Cybernetics, 50(4):1405-1417, April 2020. ISSN 2168-2275, doi: 10. WTTTCC/..001.8666611 document/844 /845n2///4822422 Conference Name: IEEE Transactions on Cybernetics.
URL https: weeeepeeeeeeeeeeeee ieee.
Yuan Yin, Ibrahim Ayed, Emmanuel de Bézenac, Nicolas Baskiotis, and Patrick Gallinari. LEADS: Learning Dynamical Systems that Generalize Across Environments. In Advances in Neural Information Processing Systems, volume 34, pages 7561-7573, Curran Asso- ciates, Inc., 2021. URL hettpa::/pproceeddng: neurips cctccc/ppaaa http//apee/2221//1// /aah//hhhhh/// -tt aabssract.html..
Li Yingzhen and Stephan Mandt. Disentangled sequential autoencoder. In International Conference on Machine Learning, pages 5670-5679, PMLR, 2018
Peter Zeidman, Amirhossein Jafarian, Mohamed L. Seghier, Vladimir Litvak, Hayriye Cagnan, Cathy J. Price, and Karl J. Friston. A guide to group effective connectivity analysis, part 2: Second level analysis with PEB. Neurolmage, 200:12-25, October 2019. ISSN 1095-9572. doi:
www.:/faeerormmmc2000000 13..0.0033.
Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitik. Self-supervised contrastive pre-training for time series via time-frequency consistency, 2022. URL htttttt/p//xiiii org/ abs/ abs/2206 08496.
O. Zoeter and T. Heskes. Hierarchical visualization of time-series data using switching linear dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10): 1202-1211 1214, October 2003. ISSN 1939-3539. doi: 10.1 199/0///22M 100...28....1.895.... URL https: // //ieeexplore, feee. org/ pocument/123338 Pattern Analysis and Machine Intelligence.
Conference Name: IEEE Transactions
16

---

Published as a conference paper at ICLR 2025
A
METHODOLOGICAL DETAILS
Training data
Individual DSR model
Training algorithm Seee (N.)AA) dG (K)
Gruc (fP) e
cupjert]
Time
Suggect-aeechcc
Oroap-lensi paramaseera Rgowp
Sutgect-spolte DSM model
6. Parameter updales (Adam, SGD.)
*
everage acroes poncessnnrstt
Loss per training sequence
5. Gradiants chain rule (eutograd)
au =(4" gin (600 -200-p7
Figure 8: Illustration of the training algorithm for one example training sequence.
A.1 MODEL TRAINING
A.1.1 GENERALIZED TEACHER FORCING (GTF)
GTF interpolates between forward propaggted latent states 2, of the DSR model and states žn h- (zx) inferred from the observations according to:
&, :1 a))z + a,2,
with 0 S S 1. which may be set by line search or adaptively (automatically) adjusted throughout training (here we simply picked reasonable defaults based on Fig. S3 Hess et al. (2023)). Hess et al. (2023) prove that optimal choice for leads to bounded gradients sequence length T goes to infinity. An extension of GTF to multimodal and non- non-Gaussian data (multimodal teacher forcing, MTF) has recently been proposed (Brenner al., 2024). MTF can be naturally incorporated into hierarchical framework and hence is an interesting avenue for further applications.
A.1.2 DITERENTIAL LEARNING RATES
For training we used the RAdam (Liu et al., 2020) optimizer. To ensure successful training, we found it crucial to assign significantly higher learning rate to the subject-specific feature vectors (10-3) than to the group-level matrices (10-4). This helped prevent numerical instabilities in the group-level matrices, which occurred with higher learning rates. It also prioritizes the incorporation of subject-specifte information through the feature vectors, see Fig. 19.
A.1.3 INITIALIZATION
We used Xavier uniform initialization (Glorot and Bengio, 2010) for the group-level matrices, which designed to keep the variance of the outputs nout of a layer roughly equal to the variance of its inputs
nin by drawing weights from a uniform distribution in the interval [-a, a). where a = toow We reduced the weights further by a factor of 0.1 to stabilize training. Additionally, we applied L2 regularization to the group-level matrices to further enhance stability.
1.1.4 BATCHING
We employed a batching strategy that subsamples training sequences æ 7:++70) -1 of length T-U) from
the total time series of length T(ff), starting at a randomly drawn position T E {1, (1i)) T + 1}. Given the complexity of the datasets considered here, an optimal data loading strategy for stochastic optimization is not obvious a priori. We hence investigated how different batch sizes impact the reconstruction performance of models trained on the 64-subject Lorenz-63 dataset (see main text). The blue curve in Fig. illustrates how performance varies across a range of batch sizes when training with 50 batches per epoch, corresponding to fixed number of gradient updates per epoch. In this
17

---

Published as a conference paper at ICLR 2025
7-
0.42
0.40
6-
0.38
0.34
0.32
3-
0.30
16
32
64
120
256
512 1024 2048
4096
16 32
64
120
256 512
1024 2048 4096
batch
Figure 9: Mean reconstruction performance for different batch sizes (error bars = standard deviation). The blue curve corresponds to training with fixed number of gradient steps per epoch, while the orange curve represents training with fixed amount of data per epoch. Some models with smaller batch sizes failed to train entirely, leading high variance for batch sizes 32 and 64 and the exclusion of batch size 16.
setup, models trained with smaller batch sizes naturally see less data per epoch than those with larger batches. To ensure that the observed effects are not merely due reduced data exposure with smaller batch sizes, we repeated the experiment using the same batch sizes, but ensuring the entire dataset was seen in each epoch (orange curve). Our findings suggest that smaller batch sizes do not contain enough statistical information across all subjects, leading to higher variance in gradient updates and, some cases, failure of training. For larger batch sizes, the performance difference was marginal.
Dytop
DH
64
5.4
0.400
5.0
0.384
Number of subjecs
4.6
0.368
32
3.8
0.336
3.4
0.320
16
3.0
0.304
0.288
16
32
64
128
256
512
1024
2048
4096
16
32
64
128
256
512
1024 2048
4096
batch
Figure 10: Reconstruction performance for different batch sizes and numbers of subjects. The optimal batch size ranges approximately between 4-16 x the number of subjects, as highlighted by the dashed black lines.
The previous experiments were conducted with fixed number of subjects. However, since we aim to incorporate information from all subjects within each batch, we extended the analysis to examine how the optimal batch size scales with different numbers of subjects. To do this, randomly sampled smaller subsets from the dataset and repeated the experiment. Fig. 10 gives heat map of the results, showing that the optimal batch size falls between 4 to 16 times the number of subjects in the dataset. For datasets with a large number of subjects, this range becomes computationally infeasible, so we instead sample batch sequences from random subset of subjects each iteration.
A.1.5 ROBUSTNESS
As evident from Tab. 1, repeated training runs with the hier-shPLRNN deliver highly consistent results. Fig. 1lb dissects this further by plotting the median performance (across the subjects) for all ten training separately, which exhibits only little variation. We further quantified the robustness of the learned feature spaces by first calculating the pairwise cosine similarities between subject feature vectors any given run, and then the correlations among those across different training
18

---

Published as a conference paper at ICLR 2025
b
State space divergence
Hellinger distance
Feature space correlations
0.065
0.82
2
0.80
0.060
5
0.76
0.055
0.74
0.050
0./0
10
0.68
median
2 4
9
median
training run
training run
8
10
Figure 11: a: Cross-correlation matrix of cosine similarities between the S = 20 feature vectors for Lorenz-96 training runs. b: Mean performance measures across 20 subjects for Lorenz-96 training runs.
runs (yielding the cross-correlation matrix in Fig. Ila for the Lorenz-96 benchmark). Average cross-correlations for all the different settings studied here in Tab. 2.
Table Mean + SD feature space correlations for experiments.
Experiment
Lorenz-63 0.90 = 0.03
Lorenz-96 0.75 + 0.05
Pulse Wave
EEG 0.81 + 0.03
fMRI 0.85 = 0.05
Correlation
0.88 + 0.04
A.1.6 OTHER ARCHITECTURES TESTED
Besides the shPLRNN for which results are reported in the main text, we repeated the Lorenz-63 experiment in sect. 4.1 with the following RNN architectures trained by GTF (Hess et al., 2023): First, simple 'vanilla RNN'
2. tanhh 2:-1 +),
(7)
and, second, an LSTM
f. (W + 1/ 1 by)
i (W U, + bi)
0, (W Wo + U b.)
(8)
è tanh (W, - 1 bb)
C f. @ 6-11 + +200 c5
o, O tanh (c,)
where @ denotes element- wise multiplication. Since both architectures use sigmoidal activation functions, we employ linear observation model, shared across subjects, scale the latent states back to the observation range.
Results Tab. indicate that while both these RNNs also profit from the hierarchisation, performance for the LSTM is considerably worse than for the shPLRNN and vanilla RNN (potentially because due to its specialized architecture the LSTM gains less from the Gas-d training; for DSR, the training technique is actually more crucial than the architecture, Hess et al. (2023)).
Additionally, we tested how using a linear observation model, i.e. a h (z( 2 (0)
influences hier-sPPLRNN performance. We tested both, using shared observation matrix across all subjects (B B Vj), as well as subject-specific observation matrices constructed from the respective feature vectors (BG) mat(((). PR. N, M)). Results in Tab indicate that both variants of linear observation model degrade performance slightly.
19

---

Published as a conference paper at ICLR 2025
Table 3: Median (#: MAD) reconstruction performance for the two alternative RNN architectures, and for two variants of linear observation models for the shPLRNN, on the Lorenz-63 benchmark from Sect. 4.1.
Architecture
sasp
DH
ensemble
1.72 + 0.24
0.296 + 0.008
Vanilla
hierarchical
0.57 + 0.09
0.199 + 0.010
ensemble
2.7 + 0.3
0.76 + 0.017
LSTM
hierarchical
2.3 + 0.3
0.37 0.06
hier-shPLRNN
0.55 + 0.26
0.087 0.012
+ shared obs. model
hier-shPLRNN
0.54 +: 0.18
0.087 + 0.010
hierarchical obs. model
A.1.7 HIERARCHIZATION SCHEME
As discussed in Sect. 3, our hierarchization framework is flexible, allowing for different forms of the group-level function Go that maps subject-specific feature vectors model weights. Here we discuss different choices for parameterizing Ge and points to consider in this context. The naive approach, employing direct linear projections. results over-parameteriiation. For example, the weight matrices W1 and W2 of the shPLRNN have M x L degrees of freedom, but when parameterized with feature vectors through w mat(((). Pw,, M, L). the number of trainable parameters expands to Niea x (M X L + 1), which with Neat > 1 exceeds the M L available entries in W1.
To address this, we tested using outer product forms, reducing parameter load by constructing all subject-specific weight matrices from two matrices Pw; E IR XNaa and Qw, R New via:
w Pvr, diag (70))
(analogously for W2), while the diagonal of and vectors h1, h2 were specified directly through linear maps. This reduces the number of parameters Nia x (M T, + 1). In the cases where Nicat < M, such that the matrices would not be full rank, we applied an extra step to map the feature vector to an auxiliary vector M (5)) = RI6) via the shared" matrix R IR. 1gMxNN XNee, and
subsequently mapped to the weights w Pw, diag(a (Qw (analogously for W2). We also tested using an MI.P for mapping the feature vectors onto the subject-specifie DSR model weights, with MLP parameters shared among subjects:
0 = MLP((())
(10)
The MLP mapped onto flat vector, which was reshaped into the respective shPLRNN weight matrices.
Both of these schemes did, however, not significantly improve reconstruction results the Lorenz-63 dataset (see Sect. 4.1) when compared to the naive linear approach according to one-sided t-tests, see Tab. 4.
Table 4: Median state space divergence and Hellinger distance different parameterizations of the hierarchization scheme, for 10 models each trained on the 64 subjects created via the Lorenz-63 system. The MLP parameterization was clearly inferior, while the linear and outer product approaches were statistically indistinguishable.
D sbp
DH
Scheme
params
Value (1)
((18)
Value (1)
1(18)
Naive linear
6912
0.394 1 0.014
0.097
0.005
Outer product
3348
0.36 + 0.03
0.0986
0.0025
-0.63
0.27
MLP
36713
0.85 - 0.09
7755
0.0
0.111
0.004
-5.54
0.0
20

---

Published as a conference paper at ICLR 2025
sttpp
DH
1.0
1.0
hierarchical baseline
0.8
0.8
0.6
0.7
0.4
0.6
500
1000
1500
2000
2500
3000
500
1000
1500
2000
2500
3000
Tmax
Tiax
Figure 12: Relative performance (hicrarchical/baseling) as function of time series length for the 64-subject Lorenz-63 dataset (lower better).
A.1.8 PERFORMANCE AS A FUNCTION OF DATA SIZE
To further evaluate how performance scales with the number of subjects and available time series length, we conducted additional experiments on the Lorenz-63 dataset as in Sect. 4.1, but with varying number of time steps for each subject/system, where 1000 training time steps corresponds to the setting in Sect. 4.1. Fig. 12 shows the ratio between the hierarchical model's 's performance and the baseline performance (i.e., training ensemble of individual shPLRNNs, with values below 1 indicating superior performance of the hierarchical model). Even for long training sequences the hierarchical model consistently outperforms the baseline, particularly in terms of state-space divergence.
We furthermore evaluate how the performance depends the number of subjects S for different values of time series length T max. using at least two different dynamical regimes (limit cycle and chaos) for the Lorenz-63 (hence starting gains are substantial up to S = 16 for this dataset.
2). Results are in Fig. 13, illustrating that performance
0.8
0.7
0.5
Dasp
8-
0.4
4-
0.2
2
0.1
200 400
600
800
1000
200
400
600
800
1000
trajectory length per subject
trajectory length per subject
I 2 subjects
subjects
I
32 subjects
b 4 subjects
16 subjects
64 subjects
Figure 13: Dependence of model performance on the number of subjects S and trajectory length Tmax-
"To avoid increasing the number of free parameters, R could also be chosen be random and constant.
21

---

Published as a conference paper at ICLR 2025
A.1.9 COMPUTATIONAL COMPLEXITY
Qua construction, inference times (i.e., forward-iterating the full model from some initial condition) should scale linearly with number of subjects S and number of features Neat, which Fig. 14b-c confirms they do (note that due construction of the subject-specific model matrices DSR initially setup times for the hier-shPLRNN are slightly larger than for an ensemble of shPLRNNs, but this difference is negligible in practice for realistic settings with Neeat << S and more than made up for by the fast split-second fine-tuning of the hier-shPLRNN once trained).
0
More importantly, as shown in Fig. 14a, actual training times for the hier-shPLRNN until a pre-set performance criterion was reached (Dasp I 1.0), actually decrease with the number of subjects S (note that all training here was intentionally performed on single CPU core, hence there was no parallelization here). The reason for this presumably is that incorporation of more subjects, and hence sampling of larger portion of the underlying systems' parameter space, allows the model to reach the criterion and reconstruct each subject's dynamics much more quickly. This further illustrates that the hicr-shPLRNN leverages and transfers information from multiple subjects highly efficiently.
150
convergence time [rain
125
Infrrecce time (mwc]
30
100
inference
40
20
20
0-
500
1000
1500
2000
ron-tierarchica Nose Nvw Mea
200
400
600
800
1000
Abor
Mes 64
Hepp 1024
Figure 14: a: Wall clock time of model convergence (i.c. first passage time for median sssp 1) for models trained with different S (using the Lorenz-63 system with p (28, 34, 40, 46, 52, 58, 64, 70, 76, 82)). For all values of S, the total number of data points was held constant 20, 000 time steps, uniformly divided among all subjects. Error bars = SEM across ten runs, sampling Dupp every 50 epochs. All models were trained on a single core of an Intel Xeon Gold 6254. b: Wall clock time of model inference (Tey = 30, M = 3, = 50) for different Nicat at S 2048. c: Wall clock time of model inference for different S and Ne
A.1.10 UNCERTAINTY ESTIMATES
To obtain the uncertainty estimates (error bars) in Figs. and c, we first drew a long trajectory with 10000 time steps from the ground truth system using (5+1), We then randomly sampled multiple training trajectories of length Tmax, inferred the feature vectors [S+1] for each of those individually, and calculated the standard deviation of the resulting 5($+1) estimates. As shown in Fig. 15, the estimates sharpen longer training sequences used.
A.1.11 HYPERPARAMETERS
In Tab, 5 we report the hyperparameters used for training the shPLRNN models on the datasets in Sect. 4.1. These include the latent dimension M (taken to be equivalent to the dimension of the observed data here, thus fixed), the hidden dimension L, the GTF interpolation factor a, and the number features Ntent As recommended in Hess al. (2023), a decays exponentially from Okann at the beginning to Qent at the end of training, where our values are based on Fig. S3 in that paper and not further fine-tuned (nor adaptively determined). The only parameters determined by grid search were thus Niat and L.
A.2 EVALUATION MEASURES
State Space Divergence Dasp is the Kullback-Leile divergence between generated, Pgen (z), and ground truth, Pirue (x)) trajectory point distributions in state space (hence measure of geometrical
22

---

Published as a conference paper at ICLR 2025
b
0.4
800
0.125
Training Steps
0.100
0.3
500
Density
Loss
0.075
0.2
300
0.050
100
0.025
30
0.0
40
50
60
70
80
0.000
20
40
80
100
Predicted
Predicted
Figure 15: Distributions p(p()
x ())
T...Tam. (approximated using kernel density estimates), estimated
from 10 time series each, with different lengths mix (color-coded) for the example Piest from Fig. 6. b: Corresponding GTF loss curves for different training sequence lengths, illustrating that longer sequences/more training data lead to smaller standard deviation over loss curves.
(f)
Table 5: Hyperparameters for our models from Sect. 4.1.
Benchmark
Model
Niea
M
L
Ostar
Ckend
shPLRNN ensemble
0.2
0.02
Lorenz-63
hier-shPLRNN
3
150
0.2
0.02
shPLRNN ensemble
10
100
0.2
0.02
Lorenz-96
hier-shPLRNN
10
200
0.2
0.02
similarity):
Pgea(x) (x) log Pgen (x)
Dsssp LK ((PPeeellPuue)
dr
(11)
Purcc (I)
In lower dimensional state spaces this may be approximated by discrete binning: Let be the number of bins in each dimension and p(*) the number of points in the k-th bin, then with a total number of bins K md. an estimate of the state space divergence is given by
Dasp
Σ (x)
K
p(k) gen
log
Pgen
(k)
(z)
(12)
Puue
(I)
k=1
Since the total number bins seales exponentially with the observation dimension N, for experi- ments with the Lorenz-96 system, used 5 per dimension to ensure the approach remained computationally feasible. For very high-dimensional spaces, Brenner et al. (2022); Mikhaeil et al. (2022) suggest GMM-bbssed estimate with Gaussians centered the trajectory points (which was not required here, however). For the values in Table 1, we sampled single time series with 10, 000 time steps from both the ground truth system (serving the test set) and from each trained model, inferring only the initial state from the first time step of the test set, and cutting off transients (1, 000 time steps) focus on the long term attractor behavior.
Average Hellinger Distance To assess long-term temporal agreement, we calculated the average Hellinger distance DH between the power spectra of true and generated time series (Mikhaeil et al., 2022). Let fi(w) be the power spectrum for the i-th dimension (time series variable), then the Hellinger distance averaged across dimensions defined as
DH = Σ / (tree) ((trte)(w)) (w) f((on) (w)dw
(13)
The respective power spectra are estimated by the Fast Fourier Transform (FFT), which leads to the integral approximation
DH
Σ S (truc)
(w))){(8 (w)))
(14)
Test and generaege trajectories were computed for D Dasg.
23

---

Published as a conference paper at ICLR 2025
deneeddeeeentt Pp O8 Aroup 0t ()
subjjcl-speciffe parameters tin
time-dependent perameters F
a
group
Time index
28
p= 68
Time
Predicted P
60
40
30
35
40
45
50
55
60
65
70
Ground Truth p
Augmenting the hierarchical shPLRNN for identifying drivers of non-autonomous dynamics: Making system-specific features time-dependent. (a) A multi-layer perceptron (MLP), mapping time and subject-specific feature vectors (5) onto a time-dependent feature vector i(") is jointly trained with the hier-shPI.NN resulting in explicitly time-dependent parameters of the DSR model. (b) To test this idea, we simulated simple setup where the control parameter p of the Lorenz-63 system was linearly increased in the range [28, 68). mimicking time-dependent driving of the underlying system. For purposes of illustration, a hier-shPLRNN of just one subject was trained time series segments from this simulation, but the approach could easily be extended to multiple subjects. (c) The temporal drift in was successfully recovered by the hier-shPLRNN, with linear relation between actual and predicted (R 0.996). This simple example provides a proof of concept, but requires more detailed study to ensure this is viable approach for more complex underlying systems and time dependencies control parameters.
Figure 16:
A.3 DATASETS
Lorenz-63 system The famous 3d chaotic Lorenz attractor (Lorenz, 1963) is probably the most widely used benchmark system in DSR. Its governing equations are given by:
á oly x),
(15)
ý = x 2) - y:
my Bz.
Standard parameter settings putting the system into chaotic regime are a 10, 28, and 8/3. We further included Gaussian observation noise term, adding 5% of the subject-specific data variance each time series a (1) T.."""""
Rössler system
The chaotic Rössler system (Rössler, 1976) is defined by:
24

---

Published as a conference paper at ICLR 2025
á = - 2,
(16)
ay,
1 + z(x - c).
Standard parameter settings that place the system chaotic regime are 0.2, b 0.2, and 5.7. As for the Lorenz-63, we included a Gaussian observation noise term with 5% of the subject-specific data variance.
Lorenz-96 system The Lorenz-96 96 is extension of the Loren-63 system to arbitrary number of spatial dimensions N, introduced Lorenz (1996). The governing equations are defined by
: (#t+1
I2 1:-2) 24 F
(17)
with r-1 IN-1, To NN, N1+1 21. and a forcing parameter F. Again, added Gaussian observation noise with 5% of the subject-specific data variance.
Trajectories for all three benchmark systems generated by numerically integrating the ODEs using the odeint function in scipy. integrate which implements the Isoda solver with a step size of At .01.
Double Scroll (Chua's Circuit) The double-scroll attractor, often studied with Chua's Circuit (Chua et al., 1986), is 3-dimensional chaotic system. Its governing equations are given by:
i a (y - x - f())).
(18)
ÿ y + 2,
1 = By,
where the piecewise-linear function (xx is defined as:
1 f(x) m +
2 mmo m1) (|x + 1| - -1).
(19)
Standard parameter settings for chaotic regime are a = 9.0, B = 14.0, ma = -.143, and mi -..144 For the results in Fig. 28, chose [8.4,9.4] leading to both single-scroll and double-scroll behavior.
Pulse Wave Dataset The pulse wave dataset (Charlton et al., 2019) was taken from https: //peterhcharl wwwchaartonnngchhieee git 1o/ h//pwbb/.... / It consists of pulse wave time series, simulated for 4,374 healthy adults aged 25-75 years, at 13 common arterial sites within the cardiovascular system (AorticRoot, ThorAorta, AbdAorta, IliacBif, Carotid, SupTemporal, SupMidCerebral, Brachial, Ra- dial, Digital, Commonlliac, Femoral, AntTibial). For each site, four physiological quantities (Arterial Pressure Wave, Arterial Flow Velocity Wave, Arterial Luminal Area Wave, and Photoplethysmograal Wave) were simulated. This resulted total of 52 simulated pulse waves (4 quantities recorded 13 sites) per virtual subject. The dataset further contains 32 hemodynamic parameters used for simulating the subject's pulse waves, listed in Table 6. The parameter PFT (time of peak flow at the aortic root) was nearly constant (78 for 124 subjects and 80 for all other subjects) and hence omitted from our regression analysis. For training, we standardized the time series subject- and dimension- wise, since some time series featured significant differences between subjects in their first moment, particularly in the Arterial Luminal Area Waves (possibly overriding some of the more relevant DS features as evidenced in lower training performance).
Human EEG Data The BEG recordings from epileptic patients and healthy con- trols were originally provided in Andrzejak et al. (2001) and reformatted by Zhang et al. (2022). The dataset is publicly available on the Time Series Classifica- tion Website (https:/ /tt///wwww. www.:timesse timeseriesclaseee wasssiiciiiion..ommmmciiio com/ descriptsii php? Dataset=Fp 1 lepsy2) under 'Epilepsy2". The original set contains 500 single-channel EEG
25

---

Published as a conference paper at ICLR 2025
Abbreviation
Parameter
age
Age, years
EI
lleart rate (IIR) beats minute (byen)
Sv
Snoke volume (SV), millilitres
CO
Cardine qunpu (CO),
[[,min)
LVET
ventiiulaar ejection time (LVET), milliecoons (ms)
dalaa
Max. value of first derivalive the nottic nool pressure ware (mmligh)
PFT
Tame of peak flow at aomi roce, in milliseconds (ms)
RFV
Reverse flow walume sortc root, in milltitres (mil
SBP
Systolic blood pressure, minliz
DBP
Distelic blood pressure, mmllg Mean blood pressure
MBP
PP
Pulse pessurre
PP amp
Pulse peessure amplification ratio beachial aurtic Augmenation pressure at the carofid artery, in mmilg Augmeszation index at the carota antery. 16 Time to reflactad wave at conolid intory, in ms Pulse wore velocity (mV4) messured berween the soctic root and illne bifureation
AP
Aix
Tr
PWy
PWV of
Pulse velocity (m/s) messured betwern the cunotid und femoral aneries brachial and radial
Pw
Pulse velocity
PWV
Pulse
femoral
tibial (anklke)
Ascanding diameter (mm)
Descending thoracic seeth diameter (mmm)
dia abda
Abdominal aora diameter (mca)
dia car
Carold diametes (mm)
las
Length of prosinall aorta (onm)
drop fin
MBP drop from weriic noot to digital artery, mmift MBP drop from acctic
aanile
tibia artery mmilg
SVR
Systemic
resistance Pas/mm)
Table The 32 ground truth haemodynamic parameters for the pulse wave dataset (Charlton et al., 2019). Descriptions taken from pwdb ha emoood_a..... Lamnod_praann params csv.
cww. con/petterhcharlton) pwwp/wwkk/
measurements, recorded for 23.6s at 174Hz. These were then split into 1s sequences, which were labeled as displaying either epileptic or healthy activity. From these sequences, 80 (40 per class) were selected as the training set on the Time Series Classification Website, which we used for our experiments. We further standardized the data with a global mean and standard deviation across all subjects in order to retain the relative amplitudes of the sequences.
Human fMRT Data The fMRI dataset analyzed in this study was initially collected by Koppe et al. (2014), re-analyzed in Koppe al. (2019), and made publicly accessible as part of the latter work. It comprises time series from 10 brain regions bilaterally, chosen for their consistent involvement in n-back working memory tasks as identified in meta-analysis by Owen et al. (2005). Eigenvariates for these regions were extracted using the Brodmann area (BA) atlas. Specifically, the regions included medial and inferior parietal areas (BA7, BA40), prefrontal regions such the dorsolateral prefrontal cortex (BA9, BA46) and ventrolateral prefrontal cortex (BA45, BA47), as well as the frontal pole (BA10), supplementary motor area (BA6), dorsal anterior cingulate cortex (BA32), and the cerebellum.
Electricity Load Data The dataset from Trindade (2015) consists of electricity load time series from 370 sites recorded between 2011 and 2014. Each time series was preprocessed by removing leading zeros to account for differing start times across sites, resulting in varying maximum time lengths () The time series were standardized and slightly smoothed using a Gaussian kernel (window size = 5) and delay-embedded with a dimension of d = 5 and a lag of 10. We trained a feature vector with Niaa 12 for every site.
A.4 COMPARISON METHODS
Selection of comparisons The approaches by Roeder et al. (2019), Yin et al. (2021) and Kirchmeyer al. (2022) are the most sensible benchmarks for our study, as they address both forecasting and generalization across different parameter settings of DS. Yin et al. (2021) and Kirchmeyer et al. (2022) were directly evaluated in our work, while Roeder et al. (2019) was excluded due to their focus short-term dynamical responses with relatively simple dynamics, and the integration of prior knowledge. Inubushi and Goto (2020) and Guo et al. (2021) do not learn subject-specifis models or focus solely on shared dynamics, limiting their relevance for DSR. For Bird and Williams (2019), the primary focus was not on DSR but rather on generating sequences in specific styles using a multi-task DS. Further, no code was publicly available. Desai al. (2022), the other hand, requires strong
26

---

Published as a conference paper at ICLR 2025
physical priors, making it less suitable for our benchmarks, which focus more general, data-driven methods.
LEADS
The LEADS (LEarning Across Dynamical Systems, (Yin et al., 2021)) framework decom- poses the dynamics model into shared and envirofment--pecfffi components. For each environment E E, the dynamies are modeled as linear combination of shared dynamics f and environment- specific dynamics ge:
fz() = f(x) + (x)),
(20)
where f captures common dynamics across environments, and ge models the individual characteristies of environment e. The learning objective is to minimize the complexity of the environment-specinn terms Je while ensuring accurate modeling of the overall system dynamics. This is achieved by
solving the following optimization problem:
(gge) subject to Vei, dxf
min J.ER
f(x + (x
(21)
where (g.) a regularization term that penalizes the complexity of the cwvironment-speciffe terms.
We used the implementation of LEADS provided by the authors on their public GitHub page (https: /ttg://gttttt wwwm yuann SeeaanyinnLAAaee LiADLE For both the Lorenz-63 and Lorenz-96 bench- marks described in Sect. 4.1, we scanned different sequence lengths (seq_len E (5, 10, 20, 30), using a fixed time step dt = 0.01 and adjusting num_trai_per _nv to provide the same number of training time steps per environment/seting as for the other methods. We used the Forecaster net with the 'mlp' setting and a hidden dimension of 6/20 (for Lorenz-63/96) to ensure upproximately the same number of parameters as for the hier-shPLRNN, We used the default learning rate of Ir = 1.e - 3 as employed for all experiments in Yin et al. (2021), and scanned sambd__inv E (5.e- 4, I.e - 4, 1.e 1.e-5) 5) and facter_lip E (1.c- 1.e 1.e -4.. Despite hyperparameter tuning, LEADS struggled to capture the long-term dynamics of both the Lorenz-63 and Lorenz-96 systems. Models either converged divergent behavior for long-term forecasts. Given that performance metrics cannot be computed for divergent trajectories, the results presented here are hence for models predicting fixed point dynamics after short transients locally fitting the dynamics.
simple fixed points or exhibited unstable and
CoDA As in LEADS, in CoDA CContext-Informct Dynamics Adaptation, Kirchmeyer al. (2022)) the dynamics for each environment E E is modeled as combination of shared dynamics f and environment-specific dynamics. Specifically, a low-dimensional context vector & is used to generate parameter variations via a hypernetwork, resulting in environment-specifee parameters:
0. = 0. + We.
(22)
where 0g are the shared parameters, W is learned weight matrix, and & the environment--peccfic context vector. The dynamics for each environment e are then given by:
fa(x) = f (x; 0.) f(;; 0. + W..).
(23)
As for LEADS, the learning objective is to minimize the contribution of the context vectors & while ensuring accurate modeling of the individual system dynamics. This achieved by solving the following optimization problem:
[(U(()) W&) + + min f.W. (EL) eEE
(24)
where 122 is a regularization term. We used the implementation of CoDA provided on the authors' GitHub site (htsps://gith) thub. com/ www.//////muuuaeee yuan-yind /CDAA) The architecture is fixed, allowing only to change the various regularization coefficients as hyperparameters. For those we scanned similar values as the authors in their experiments, xe E {
[1.e- - 2, I.e 3, L.e - 4}.
Xr, {L.e- - 5, L.e 6, 1.e - 7} dτ, {Lee- 5, L.e 6, L.e 7}. and dim(E) E [3, 6, 10). While overall working much better than LEADS, CoDA was still less reliable in recovering the long-ter term properties of the underlying systems than method. Fig. 17 illustrates this for an example subject trained on the Lorenz-63, where the power spectrum learned by CoDA is much less accurate, especially in the low-frequency range.
27

---

Published as a conference paper at ICLR 2025
hierarchical shPLRNN
CoDA
10 2
10
power density
10
10
10
10-7.
100
10a
102
10a
10*
10
10a
10a
10a
10a
frequency
frequency
Figure 17: Power spectra ground truth (blue) and generated (red) trajectories of one of the Lorenz- 63 settings for hierarchical shPLRNN and CoDA.
A.5 UNSUPERVISED FEATURE EXTRACTION
tsfresh We used the Python implementation fresh (Christ al., 2018), which automatically extracts a large set of time series features and selects the most relevant ones for further analysis.
Catch-22 Catch-22 a collection of 22 time series features that are both informative and com- putationally efficient (Lubba al., 2019). We used the default implementation from the tslearn package.
ROCKET and MiniRocket ROCKET (RandOm Convolutional KErnel Transform), and its variant MiniRocket, random convolutional kernels project time series into high-dimensional feature spaces useful for linear classification and unsupervised feature extraction (Dempster et al., 2020; 2021). Here used the default setting of 10, 000 random kernels in the tslearn package, leading to 20, 000-dimensional embedding space.
Attention-based Convolutional Autoeneoders Lastly, we trained an attention- convolutional autoencoder (Wang et al., 2023), alternating convolutional layers with time and feature attention layers in sequential order. We fitted a GMM for unsupervised cluster assignment to the latent representations learned by the encoder, where chose the size of the encoding space be the same as that of the hier-shPLRNN feature space (Niest 10).
Table 7: Average classification accuracy of Gaussian Mixture Models (GMMs) applied to features extracted through various unsupervised methods on the EEG data. Values are across 10 training runs for each model. A randomly initialized GMM was fitted for each training run, so variation in deterministic models stems from the different GMM fittings, while in noon-detemministcc models' (hier-shPLRNN, MiniRocket, Attn-Conv-EE) it comes from both training algorithm and GMM fitting.
Model
Classification Accuracy
hier-shPLRNN
92.6 1.2%
ROCKET (Dempster 2020)
2.2%
MiniRocket (Dempster al., 2021)
65 :: 13%
Isfresh (Christ et al., 2018)
68.1 + 3.3%
Catch (Lubba 2019)
57.6 2.5%
Attn-Conv-AE (Wang al., 2023)
71.5 + 1.8%
A.5.1 HIERARCHICAL MARKOV NEURAL OPERATOR
To illustrate the wider applicability of our approach, also tested the hierarchization framework with the Markov Neural Operator (MNO) suggested in Li et al. (2022). Code is publically available at https: : //githuu com/ htwp//eperaaatoooo /aaroov neural opwrator. As described in Sect. 3, we employed www-dimessionl feature vectors and higher-dimensional group-level matrices linearly mapping onto all the trainable parameters of the MNO. We trained with N feat on the
28

---

Published as a conference paper at ICLR 2025
Lorenz-63 setup from sect. 4.2, and batch size, learning rates and initialization chosen discussed in Appx. A.1. Following Li et al. (2022), we trained on one-step-ahead predictions, combined with physics-informed dissipative loss. This loss enforces dissipativity by penalizing deviations from the global attractor through a regularization term that mimics energy dissipation. This initial implementation and the results in Fig. 18 meant to highlight the straightforward applicability of the hierarchical framework to other architectures.
b
Ground Truth Generated
a
Predicted
60
30
40
Ground Truth
50
60
70
80
= 24
=
80
Figure 18: a: Ground truth and predicted p from 1d feature values for the hierarchical MNO model (linear fit: R = 0.97), similar to the results in Figs. & 5. b: Long term freely generated dynamics for different p values.
29

---

Published as a conference paper at ICLR 2025
B
FURTHER RESULTS
Mean Parameter Change
0.20
0.15
(9]
Pwi Parameter
Pmi
Figure 19: Average parameter changes during training of hierarchical model on time series of the Lorenz-63 system. Subject-specific feature vectors (1()) change much more than group-level parameters (PA, PW1 Pway Phyy Ph2)
0.33
0.32
3.0
0.31
2.8
0.30
0.29
0.28
7
8
3
4
5
6
7
8
Nrat
Nese
Figure 20: Reconstruction performance function of Niear for models trained the 64-subject Lorenz-63 dataset. The feature vector must be large enough to encode the variation subject-specific dynamics when changing all three of the Lorenz-63 bifurcation parameters (but may not be too large either).
p 21, B=3 p= 51, 0=
p= 0=8, == 21, 0= 9, ==1
Figure 21: Ground truth (blue) and hier-shPLRNN generated (orange) trajectories for the Lorenz-63 system. Shown four example parameter combinations, illustrating different dynamical regimes of the system. All of these were generated by the same hier-shPLRNN model.
30

---

Published as a conference paper at ICLR 2025
21
51
81
111
0.6
0.6
0.6
PC4
0.2
0.2
0.2
0.0
0.0
0.0
-0.75
-0.50
-0.25
0.00
0.25
-0.75
-0.50
-0.25 PC3
0.00
0.25
-0.75
-0.50
-0,25
0,00
0.25
FC3
PC3
Figure 22: Same as Fig. 4 for the third and fourth principle component.
200
50
0-
2
Figure 23: First PC of the 5d feature vector of a hier-shPLRNN, trained on the Lorenz-96 system in
Sect. 4.1 aligns with the ground truth forcing parameter used to generate the training data.
31

---

Published as a conference paper at ICLR 2025
Bradieel Artery Attra hrssure Warn Groun Tuth
Carceed Artery Arteral Nhe Weedly Weve Groudd Batt
143
Pimaing]
103-
0.3
0.4
0.6
0.7
0.2
0.3
0.4
Time [4]
Time [S]
Freely Drenarchicll
140
P|mmiig]
103
0.3
80
0.0
0.1
0.2
0.3
0.4
0.5
0.6
37
0.0
0.1
0.2
0.3
0.4
0.5
0.6
07
10
10
Subject
Subject
d
1n-5 Seperce tempors
Digital fntery Phstrpplttthyeeeeomm Waen, Cround Trut
[00n7]
0.75
1.10
0.50
0.25
1.05
0.00
Time 1s]
16-5
Freely Generate (heeaccics 29P1 (NN)
Fverly Gerented roadica SRN)
1.00
1.15
(10m7)
2.10
0.50
0.25
3
10
2
9
10
Subject
Subject
Figure 24: Ground truth (top) and freely generated (bottom) arterial pressure wave (a), arterial flow velocity wave (b), arterial luminal area wave (e) and photoplethysmograa wave (d), simulated at different anatomical sites, for 10 representative subjects (selected by k-medoids, color-coded).
32

---

Published as a conference paper at ICLR 2025
Train hierarchical model on multiple subjects
Ground Truth Generated
Subject
a
hampy wa pontan
Subject
Subject 14
m pampon par
Subject 15
Subject 17
Fine-tune only subject-specific parameters on unseen subject
b wwwmmmyy mmham
Subject 6
Figure 25: Example reconstructions of several subjects from the experimental fMRI dataset using hier-shPLRNN with Niat 10, L 300, and training by GTF with a 0.1.
33

---

Published as a conference paper at ICLR 2025
1.00
Extracted Feature Vectors
0.99
0.98
15
0.97
Subject
0.96
14
0.95
0.94
17
0.93
Subject
b
Cluster 1 Subject 10
Cluster 2 Subject 14
BOLD Signal
2.5
Signai
www
0.0
-2.5
Cluster 1 Subject 15
Cluster 2 Subject 17
GOLD Signal
2.5
Signal
2.5
0.0
-2.5
-2.5
100
200
300
100
200
300
C
Explained Variance Ratio
0.3
0.1
0.0
Principal Components
Figure 26: Cosine similarity matrix based the average similarity of the feature vectors
20 training runs for 4 example subjects. b: Extracted similarities and cluster labels reflect visual differences in the recorded BOLD signals. Cluster assignments were consistent across all 20 runs for all subjects except one (subject 7), for whom the cluster assignment alternated between both clusters with 50/50 split. Explained variance ratio for the principal components obtained from PCA on the subject feature space for a model trained on the 10 fMRI subjects.
34

---

Published as a conference paper at ICLR 2025
DS feature space
a
1 day test set forecast
b
0.4
Cluster
Cluster 1, Site 50
Cluster 2
200
Cluster
Data Prediction
0.2
ValuelkWV)
PC2
-0.2
Cluster 2, Site 346
-0.4
-0.4
-0.2
0.0
0.2
0.4
Data
PC1
10
Predictin
Training data (excerpt)
Value (RW]
100
Site
Cluster 3, Site 100
Value [KW]
80
Site 346
800
Data Prediction
400
Value [KW]
1500
1000
Site 100
200
500
2014-12-31 00.00
2014-12-31 06:00
2014-12-31 12:00
2014-12-31 18:00
2015-01-01 00:00
2014-12-1 00:00
2014-12-08 00:00
2014-12-1500.00 00:00
2014-12-22 00:00
Figure 27: Visualization of the first PCs of the 12d inferred DS feature space for the electricity load data. Points represent sites, color-coded by cluster. Highlighted sites (black dots) correspond examples shown below. Training data excerpts from three example sites illustrate pronounced differences in time series and highlight the highly nonstationary nature of the data. b: One-day test set forecast for each representative site, comparing ground truth (red) and model predictions (blue/orange/green)) Predictions sometimes closely match ground truth values while capturing site- specific variations in complexity.
35

---

Published as a conference paper at ICLR 2025
a
single scroll
Explained Variance Ratio
0.4
PC2
0.3
0:
double scroll
0.2
Lorenz-63
-1
Rössler
0.1
Chua's Circuit
0.0
-2
2
PC1
PC2
PC3
PC4
PC5
Principal Components
PC1
c
Ground Truth Generated
single scroll
double scroll
Figure 28: a: First 2 principal components (PCs) of the DS feature space for a model jointly trained on observations from the Lorenz-63 system, Rössler system, and Chua circuit, with 10 different values each for the GT parameters p, c. and a. The different DS are clearly separated along PCI, while PC2 distinguishes between the single-seroll versus double-scroll behavior of the Chua circuit (see e). b: Despite joint training three different DS, the hier-shPLRRNN embeds features in low-dimensional space where the first four PCs capture almost all variation across subjects. c: Example reconstructions for all three systems.
36

---

Published as a conference paper at ICLR 2025
Short-term forecast
b Geometric Agreement
C
Agreement Agreeeel
N-HITS
4003
1000
5006
Astoccretti meteeeeettooo
Transformer
5000
5400
50
100
120
200
250
300
Autocoralatton
Steps
300
4800 4000
3000
3320
3400
Staps
1000
150 300
Figure 29: Comparison several popular time series (TS) models, N-HITS (Challu et al., 2022), Transformers (Vaswani et al., 2017), and Temporal Convolutional Networks (TCNs) (Bai et al., 201 8), to the shPLRNN trained with GTF (Hess et al., 2023) with respect to forecasting performance (a) and long-tern statistics (b, c) on observations (Tmaz 5000) from the chaotic Lorenz-63 attractor (Eq. 15). TS models were trained for 500 epochs using their implementations in the darts package (Herzen et al., 2022), with input window size of 100 time steps, to forecast 30 time steps. All TS models had around 40, 000 parameters, while the shPLRNN only required 706 trainable parameters in the chosen configuration (M 3, L 100). Long-term forecasts the shPL.NN- - by recursively feeding the TS models their own predictions. a: Short-term test set forecasts generated by the TSF models often match (or even exceed) the accuracy of the shPLRNN. The somewhat better forecasting performance is easily explained by the longer initialization windows used for the TS models compared the shPLRNN (and potentially the much larger number of trainable parameters), which is initialized using just single time step Ig at the beginning of the test set (in line with state space's 's Markov property in DS theory). b: However, unlike the shPLRNN, TS models fail to reconstruct the Lorenz-63 attractor geometry. c: Likewise, and unlike the shPLRNN, their long-term forecasts completely fail to capture the statistical temporal structure of the data, assessed here via the autocorrelation function (Wood, 2010) over the first 300 time lags.
generated similar to
37


Reviews:
[{"review_id": "ozmUL300i5", "reviewer": "Reviewer_LBGT", "summary": "This paper tackles the problem of analyzing multiple time series to extract the underlying dynamical properties that govern the observed data.  The authors propose hierarchical variant of the shallow PLRNN model which introduces group-level parameters in a subject latent space.  They propose a training scheme and implement/test the method on various experiments involving transfer learning and few-shot learning, comparing their method's performance with individual time series models and other models in the literature.  They also perform various analyses to showcase how their method is interpretable and uncovers ground-truth information about the underlying dynamics.", "questions": "- In Table 1, is there any sense of how your results vary as a function of T_max?  I'm wondering if there's a substantial regime of T_max (i.e. time series length) in which the group model performs much better than single time series models?  It would be interesting to see at what point performance between group/single becomes similar, if at all.  This could give practitioners a sense of what regimes the group model has an advantage over individual level models.\n- As the authors mentioned in their literature review, there is a large body of work on sparse identification of nonlinear dynamics (i.e. Sindy) and physics-informed neural networks (PINNs) -- alternative approaches that have tackled some of the same datasets (e.g. Lorenz) as those used in this paper.  Is there a sense of how well your method compares against these other approaches in the literature (e.g. in an experiment such as Section 4.1)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This was a very nice paper to read, well-organized and well-written.  The authors included an extensive related works section to cover many developments related to their work.  In general, the presentation was quite clear, the figures were nicely formatted, and the experimental details were explained well.  \n- The problem of extracting dynamical insights from many time series is a problem of both established and burgeoning interest across many fields -- particularly in scientific ones in which data is plentiful, yet we believe there are underlying laws that determine how the data is generated.  The ability to incorporate group-level information / multi-task learning into a model arguably has the potential to learn these dynamics better; it is exciting that the authors are able to show notable improvements in performance over individual-level models and some recently proposed other methods.\n- I appreciate the authors' analyses to provide further insight into why their method is able to perform well.  It is one thing to obtain better numerical performance on a task; it is another to perform well for the right reasons.  The interpretability experiments are interesting because they convey that the model has been able to capture some knowledge of the underlying physics, which is something that one cares about in scientific applications.  \n- The authors perform several experiments to showcase their method in different settings.  It appears that the method would have wide applicability to a range of different problems.\n- The authors provide a link to their code and the code appears well-organized and a useful resource for the community.", "weaknesses": "- The model development in a bit limited.  While the authors mention that their method is technically applicable to any DSR or TS model, they mainly focus on shPLRNN.  Their experiments focus on this one instantiation; it would be interesting to consider how the hierarchization could be applied to other models as well. \n- The training method is also an amalgamation of many established techniques in the literature.  Though there is limited development of any one of these individual techniques, it should be noted that combining many ideas and getting things to \"work\" in practice is also a useful contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jMSjmv5F82", "reviewer": "Reviewer_vh2v", "summary": "The paper presents a method for learning dynamical systems by leveraging information from multiple observations (referred to as ``subjects'') for scientific analysis. The authors achieve this by learning shared parameters across observations while preserving individual variability through a hierarchical model for time-series and dynamical systems reconstruction. They validate their approach using both synthetic data (e.g., Lorenz attractor) and real-world data (e.g., EEG), demonstrating that their method captures unique dynamical characteristics that other methods overlook.", "questions": "1) Regarding Eq. 4, in line 296 you mentioned for the first time that it is piecewise-linear. Did you mean it is locally linear? Or are there full periods of linearity that switch between them (e.g., as in [1] from my Weaknesses text cell)? From equation 4 itself it just seems linear so  I would clarify it near the equation itself if that is the case.\n2) Can you give some intuition for the meaning of the different $ P $s (e.g., $P_A$, etc.)? I would interpret them as capturing groups of features or parameters that operate together to link the latent and observed space.\n3) In line 185, why did you choose to train the model directly on the observations  (i.e., the $x_t = z_t$ assumption)? What do you think will be the effect if a different approach is chosen?\n4) When describing the batching, based on the notations, it seems that all samples start with the first time point (due to writing $x_1 \\ldots $ in line 191). Is this correct, or is it a mistake and batches taken from some random $\\tau$ to $\\tau + {T_{\\text{seq}}}$?\n5)  What does `2X10` refer to in row 495? Do you mean 20 regions? How much overlap is there between them?\n6)  How would you address temporal nonstationarities within your model?\n7) What is the model's computational complexity? How does it increase with the number of subjects?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "I really enjoyed reading the paper and believe it addresses an important issue in computational neuroscience, where the data often includes multiple non-simultaneous observations across subjects. Moreover, the paper is well written, and the results clearly underscore the model's capability compared to other methods. They demonstrate their approach on multiple examples, including both synthetic and real-world datasets, under diverse settings.", "weaknesses": "# Critical:\n1) While the paper was overall very well written, I believe that the Related Work (DSR section) missed discussion about DSR methods based on time-changing combinations of linear dynamical operators, including both  switching, e.g., [1], and decomposition [2] models.\n2) I also missed from the Related Work more methods that are designed or capable of leveraging multi-session information in neuroscience specifically, including both [3,4]. I believe that these should appear in the introduction or towards the end of the related work, or even be benchmarked against if possible (although an explanation of why these methods are insufficient may suffice).\n3) I think an important question that needs to be discussed is how many subjects you need for the method to succeed. For instance, if you have a total of $J$ subjects, what will happen if you leverage information from $J-1$ or$ J-2$? I would assume that performance will improve rapidly with low $J$ values and then level off as $J$ increases to higher numbers, potentially resembling a logarithmic function. \n4) A main concern of mine is the need for a discussion on the minimum overlap required between subjects in terms of dynamical modalities, channels, and time scales for the model to work effectively. While the authors show that different dynamical regimes can be modeled in synthetic data (e.g., through varying aspects of the Lorenz attractor under changing parameters), I assume that data from completely distinct dynamical processes—such as rapid interactions in neuropixels data combined with annual climate changes—or recordings from unrelated units even within the same field (e.g., neural activity from very different brain regions with zero overlap across subjects) might challenge the model’s effectiveness. I suspect there is a critical level of cross-subject similarity needed in both scales and channel identity for reliable performance. A complete benchmark may be impractical due to the sheer range of possibilities, but providing a list of mathematical assumptions or briefly addressing this in the discussion/limitations section would clarify these requirements.\n5) The model requires a lot of hyperparameters.\n\n# Minor but recommended:\n1) Line 67: I believe an em dash (--- in LaTeX) should be used here instead of an en dash (-).\n2) I would recommend that the authors more clearly state that the term `group` refers  to the observations from all subjects. Initially, I was unsure if `group` referred to some internal grouping within the data, so a brief clarification the first time you use it would help.\n3) While I understand the page limit, it does not seem sensible to place the illustration figure (Fig. 7) in the supplementary materials. This figure is important for understanding the model, and the authors would want readers to see it. I suggest making it shorter and moving it to the main text.\n4) Line 171-172: I suggest $P_{w_1} \\in \\ldots $ should be kept on a single line (i.e., by wrapping that formula in curly braces).\n5) It is unclear how robust the method is to different model initializations.\n6) The colormap in Fig. 4a seems off: the ticklabels do not align with the color change. I suggest using a legend with `Subject 1`, `Subject 2`, etc., instead of a colorbar, as that should be categorical, not sequential (unless Subject 1 is indeed more similar to Subject 2). \n\n\n[1] Linderman, S., Johnson, M., Miller, A., Adams, R., Blei, D., & Paninski, L. (2017, April). Bayesian learning and inference in recurrent switching linear dynamical systems. In Artificial intelligence and statistics (pp. 914-922). PMLR.\n\n[2] Mudrik, N., Chen, Y., Yezerets, E., Rozell, C. J., & Charles, A. S. (2024). Decomposed linear dynamical systems (dlds) for learning the latent components of neural dynamics. Journal of Machine Learning Research, 25(59), 1-44.\n\n[3] Pandarinath, C., O’Shea, D.J., Collins, J. et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat Methods 15, 805–815 (2018). https://doi.org/10.1038/s41592-018-0109-9\n\n[4]  Mudrik, N., Ly, R., Ruebel, O., & Charles, A. S. (2024). Creimbo: Cross ensemble interactions in multi-view brain observations. arXiv preprint arXiv:2405.17395.‏", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tOhVFFYuYu", "reviewer": "Reviewer_D8qe", "summary": "This paper presents a dynamical modeling approach for time series generative foundation models, focusing on scientific applications. The authors argue that a robust time series generative model requires both in-domain and cross-domain features, achieved through hierarchical dynamical systems. They frame the construction of such models as a dynamical system reconstruction (DSR) problem, training RNNs with low-dimensional task-specific and high-dimensional group-specific features. The method is supported by numerical experiments.", "questions": "* Why Eqn 3 is mathematically tractable?\n\n* Why does this tractability matter? It seems only loosely connected to the task of interest (e.g., certain physical systems' ODEs).\n\n* Could you test your proposal on common time series tasks? For example, multivariate time series datasets commonly used in forecasting literature, such as traffic, electricity, exchange rates, etc. If not, could you provide some justification? As mentioned in your conclusion:\n  > Unlike TS models, where the interest lies mainly in forecasting or TS classification/regression, in DSR we aim for generative models that capture invariants of the true\n \n  Why wouldn’t these common forecasting tasks contain such \"invariants of the truth\"?\n\n* This paper reminds me of an ICML 2023 paper: https://arxiv.org/abs/2306.06252. While the scope is different (they focus on feature extraction of time series by assuming the underlying dynamical system is a dynamical Ising model), they also attempt to interpret general time series through a dynamical systems lens. While the exact relevance is unclear, treating TS modeling as a dynamical system is not new. I am sure there are other works in this direction. It would be valuable to discuss more related literature in the ML community.", "limitations": "", "rating": 1, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Overall, this paper might present a good idea, but it seems rushed, which affects the contribution of an otherwise very interesting work. Regardless of the final decision, I hope the authors find my comments helpful in further refining their draft.\n\n* Important modeling problem for scientific AI/ML/Foundation Models\n\n* The experiments are solid \n\n* Many ablation studies", "weaknesses": "* **Clarity.** The language is overly complex and difficult to read. For example, in lines `134-145`, the authors use 10 lines for just 3 sentences, making the paragraph hard to follow. I had to read Sec 3.1 multiple times and still found it difficult to understand.\n\n* **Novelty.** The proposal lacks clear motivation and feels too straightforward to be considered novel. Most components are derived from existing work (e.g., General Teacher Forcing, PLRNN, shPLRNN).\n\n* **Practicality.** The model's parameterization is overly complex (e.g., passing learnable parameters to the next layer), raising concerns about its feasibility for end-to-end training and practical use as a time series generative model.\n\n* **Significance.** The significance is difficult to gauge due to limited clarity. Overall, it seems lacking, given the assembly-like nature of the approach.\n\n---\n\n\n### Update 11/30/2024: Changed score to strong reject due to unprofessional rebuttal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xiMXF5ny2T", "reviewer": "Reviewer_Qfhy", "summary": "The authors introduce a general theoretical problem: reconstructing a set of dynamical systems, given short observations of each system with different underlying parameters. The authors map this problem onto problems frequently arising in biophysics and neuroscience, in which one needs to train a joint model with subject-specific attributes. The authors introduce a hierarchization scheme for training recurrent neural networks, that disentangles low-dimensional subject-specific features from the general features of a shared dynamical systems model. The authors apply their approach first to synthetic dynamical systems tasks, then to an arterial pulse dataset, and finally to EEG and fMRI datasets. Beyond strongly outperforming baselines and ablations, the authors show that their approach extracts predictive dynamical features that can be used for interpretability, or even for subject classification.", "questions": "+ Eq. 5 — can you please clarify how this is used in the training loop? How and when is \\Sigma updated? Generally I would prefer some more detail here.\n\n+ Can you provide more description of how the Hierarchical training scheme works?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This paper is very clear, thorough, and well-motivated. I think this paper clearly exceeds the acceptance bar for the conference.\n\n+ The Introduction and Related Work sections are comprehensive and concise. The authors cover standard references for SciML (Koopman/PINN/neural ODE/RC), but they also highlight several hihgly-relevant but lesser-known methods, including recent works that focus specifically on reconstructing invariant measures of dynamical systems.\n\n+ The baseline models chosen are reasonable and appropriate points of comparison. The authors include an explicit ablation of the hierarchical portion of their model\n\n+ The two application tasks (EEG and fMRI) are important and provide clear motivation for this approach, which has an inductive bias for datasets with discrete subjects. Framing subject-level clinical data as analogous to training a joint model on copies of a dynamical system with different parameters is a very conceptually-rich approach.\n\n+ There are lots of novel details, which show that the authors have thought very deeply about their applications. For example, the covariance-scaled MSE (Eq. 5) helps account for clinical batch effects. \n\n+ The arterial pulse dataset is an excellent demonstration, showing that the model successfully finds a low-dimensional description of an otherwise complex and high-dimensional simulation. The regression showing that the fitted model predicts the haemodynamic parameters is a great demonstration of the interpretability of the learned features.", "weaknesses": "I’ve assigned a high score because I think this paper clearly meets standards of novelty, rigor, and interest for the conference. However, I have a few general concerns, which the authors might consider in order to attract a wider audience to their work:\n\n+ The description of the training method itself could be expanded. I understand the architecture, and the use of a low-dimensional subject vector that linearly decodes into the high-dimensional weight vectors of the RNN. However, I do not understand the mechanics of how the subject-specific vectors are found and P matrices are trained. I understand that once we have the W matrices, then we can use generalized teacher forcing; it’s the step before that where I am unclear. I see this hierarchical training scheme as one of the main contributions of the paper, and so clarifying this is essential.\n\n+ The shPLRN model is well-motivated for this problem and has been characterized extensively in prior work. However, I do perceive it as a bit of a specialized model, compared to more common approaches to DSR like PINNs or even SINDY. An example of the hierarchization scheme, which is general, used on one of these more common DSR approaches would be an interesting baseline (LEADS, a baseline currently included in the manuscript, is essentially this for neural ODEs).\n\nMy concern is that I perceive the primary contribution of this paper to be getting the hierarchization scheme, including the training approach (which may be hard to adapt to models that can’t be trained with teacher forcing). The other novelty is the choice of datasets and demonstrations, which are all interesting and strong. However, the current paper mixes these contribution with a specialized architecture from prior work, shPLRNN, making it hard to disentangle the contributions. My concern is that readers who encounter this paper will incorrectly think that the architecture itself is the main contribution, rather than the problem setup and training scheme.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)