RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Probabilistic Decomposed Linear Dynamical Systems
We build upon dLDS and propose a probabilistic decomposed linear dynamical systems (p-dLDS) model.
Rather than propagating noisy point estimates of the latent variables during inference, we improve robustness by marginalizing over uncertainty with respect to time. Additionally, we propose a tractable method for extending the dynamics model to systems with multiple fixed points.
Time-varying offset term
Note that for any parameter setting, the dLDS local dynamics (eq. ( 2 )) reduces to a linear dynamical system (LDS) that is characterized by a single fixed point centered around the origin. Yet, real-world dynamical systems often consist of much more complicated behaviors. Nonlinearities can cause a signal to navigate through multiple fixed points throughout its trajectory, while nonstationarities may change the behavior of the system entirely with new fixed points emerging or disappearing.
Simple preprocessing measures, such as mean-centering the data, are inadequate to account for these behaviors. To enable robustness against these behaviors, we introduce a time-varying offset term \bm{b}_{t}\in\mathbb{R}^{N} into Equation ( 2 ) as a flexible way to account for dynamics not readily captured by the original dLDS latent dynamics model.

Let the transition between any two state vectors \bm{x}_{t},\bm{x}_{t+1}\in\mathbb{R}^{N} be defined by the linear dynamics matrix \bm{F}_{t}\in\mathbb{R}^{N\times N} and the dynamics offset \bm{b}_{t}\in\mathbb{R}^{N} . For any \lambda>0 , the objective,

\operatorname*{arg\,min}_{\bm{F}_{t},\bm{b}_{t}}\|\bm{x}_{t+1}-\bm{x}_{t}-\bm{F}_{t}\bm{x}_{t}-\bm{b}_{t}\|_{2}^{2}+\lambda\|\bm{F}_{t}\|_{2}^{2},

is minimized when \bm{F}_{t}=\bm{0} and \bm{b}_{t}=\bm{x}_{t+1}-\bm{x}_{t} .

This result, proven in Appendix B.1 , reveals that introducing a time-varying offset term makes inference of the dynamics a degenerate problem.
While the solution in Lemma 1 minimizes the objective, it fails to capture any meaningful structure in \bm{F}_{t} as the result of \bm{b}_{t} being unconstrained. To prevent the convergence to these trivial solutions, we decompose the latent state space as,

\bm{x}_{t} =\bm{l}_{t}+\bm{b}_{t}, (4)

where \bm{l}_{t} captures fast dynamics and \bm{b}_{t} captures slow-varying trend behavior. These latent variables follow the dynamic equations \bm{l}_{t+1}=\bm{l}_{t}+\bm{F}_{t}\bm{l}_{t}+\bm{\epsilon}_{l_{t+1}} and \bm{b}_{t+1}=\bm{b}_{t}+\bm{\epsilon}_{b_{t+1}} where \bm{\epsilon}_{l},\bm{\epsilon}_{b}\in\mathbb{R}^{N} represent noise sampled from \bm{\epsilon}_{b}\sim\mathcal{N}(0,\bm{\Sigma}_{b}) and \bm{\epsilon}_{l}\sim\mathcal{N}(0,\bm{\Sigma}_{l}) respectively.
Probabilistic Time-Informed Sparsity
In decomposed models, we aim to achieve two goals simultaneously: sparsity and smoothness of coefficients over time.
Motivated by this, we incorporate dynamics-informed probabilistic structure.
First, we assume that each coefficient evolves independently of the others. Second, we introduce a hierarchical variance parameter \gamma_{t,k} that controls the sparsity for each c_{t,k} . Large values of \gamma_{t,k} encourage coefficients to be active while small values effectively shrink them to zero. Moreover, we introduce dynamics information during sparse inference by encouraging a similar active support set in consecutive time slices through the variance hyperpriors. Put together, the resulting coefficient transition density in p-dLDS becomes,

p(\bm{c}_{t},\bm{\gamma}_{t}|\bm{c}_{t-1})\coloneqq p(\bm{c}_{t}|\bm{c}_{t-1},\bm{\gamma}_{t})p(\bm{\gamma}_{t}|\bm{c}_{t-1})=\prod_{k=1}^{K}p(c_{t,k}|c_{t-1,k}\gamma_{t,k})p(\gamma_{t,k}|c_{t-1,k})

The density p(c_{t,k}|c_{t-1,k}\gamma_{t,k}) is factorized as a product of two independent priors, \mathcal{N}(c_{t-1,k},\sigma_{t-1,k}^{2})\mathcal{N}(0,\gamma_{t,k}) , where the first prior encourages dynamic smoothness while the second encourages sparsity. The second density in equation ( 5 ) is defined similarly to the hyperprior in SBL-DF
(i.e., p(\gamma_{t,k}|c_{t-1,k})=\mathcal{IG}(\xi,\xi c_{t-1,k}^{2}) where \xi weighs the influence of the dynamics when estimating \gamma_{t,k} ). The resulting graphical model is shown in Figure 1 B.
Results
We demonstrate p-dLDS in a variety of synthetic examples, highlighting improved robustness to noise and system nonlinearity. Additionally, we apply our model to a clinical neurophysiology dataset, revealing interpretable patterns where previous methods fail. We compare our method against SLDS, rSLDS, and dLDS as described in Section 2 . All datasets are split 50:50 for training and testing. Due to space constraints, we provide full descriptions of the simulation setup in Appendix C and metric definitions in Appendix D .
Simulated Motor Cortex Data in a Reaching Task
We now turn to an empirically-derived synthetic experiment related to brain-computer interfaces, where the dynamics and observation functions are nonlinear and derived from analysis of neural data.
Our focus is on the reaching task, a neuroscience experiment designed to study motor control in non-human primates [ 16 , 19 ] . In this experiment, the subjects are trained to reach towards visually cued targets, while neural activity is recorded from motor-related areas such as electromyography (EMG) data from arm muscles. Each trial consists of two distinct phases: preparation and movement. In the preparation phase, the subject plans its movement while keeping their arm still. In the movement phase, the subject physically reaches towards the target. The goal of this experiment is to decode reach intention from neural data.
We construct a dataset by first simulating a spiking neural network with known latent factors [ 7 ] trained to reproduce empirical EMG signals from the center-out reach task in [ 19 ] .
Spikes are then converted to 50-channel local field potentials (LFP) recordings via a weighted, delayed sum of synaptic currents (see Figure 3 A and B) [ 25 , 15 , 14 ] .
Our dataset contains 150 6-second trials sampled at 250 hz, where each trial represents one out of eight reach directions visually cued at a random start time.
PCA identifies that three components captures 98% of the variance. Thus, we set M=50 , N=3 , and K=4 DOs or discrete states.

Figure 3 C shows the trial-averaged DO coefficients from p-dLDS, which change smoothly and cyclically according to the true reach angle.
Additionally, the DOs appear to differentiate between the two distinct dynamical regimes, where the activity of \bm{f}_{1} and \bm{f}_{3} localize to the preparatory and movement phase respectively.
Quantitatively, we compute the linear classification accuracy of the reach angles using the average state activity over time as features (see Appendix D.5 ). Figure 3 D shows that classifiers built from SLDS features fail to capture the full continuum of reach angles.
This limitation occurs because the discrete switching states are unable to efficiently capture the diversity of activity present in the LFPs, which arise from randomness inherent in the spike sampling process. Consequently, the inferred features from SLDS generalize poorly to held-out data.
In contrast, the p-dLDS classifier predictions recover the full spectrum of reach angles since features are naturally continuous and the inferred coefficients can adjust the learned DOs to accurately capture the activity in the held out data. Table 2 shows that p-dLDS outperforms all other models in state and dynamics reconstruction as well as the top-1 and top-3 reach classification accuracy.
Clinical Neurophysiology Data
We demonstrate p-dLDS on LFP recordings from the subcallosal cingulate cortex (SCC) in patients with treatment-resistant depression (ClinicalTrials.gov identifier NCT01984710). Subjects are asked to watch videos with different emotional content (positive, negative, and neutral), describe the videos, and then discuss how the video made them feel. SCC dynamics have been previously shown to provide a quantitative signal for the presence of emotional content [ 13 ] and depression recovery [ 1 ] . Thus, we hypothesize that the underlying dynamics may provide information about emotional changes throughout the experiment. We apply p-dLDS to a single patient’s LFP spectrogram data (Fig. 4 B, E) within the 0-40 Hz frequency range ( M=40 ).
PCA indicates that the first 7 components explain 90% of the variance. Therefore, we train a model with N=7 latent dimension and K=4 DOs.

In Figure 4 , rSLDS and dLDS produces a high degree of state oscillations making it difficult to identify time intervals with consistent emotional content. In contrast, p-dLDS infers coherent structure that corresponds to changes in emotional content in the trial.
For example, \bm{f}_{4} (red) coincides with resting, \bm{f}_{2} (orange) with positive videos, and \bm{f}_{1} and \bm{f}_{3} (blue and green) to negative videos (Fig. 4 C). Importantly, this structure persists even on held out data from the second half of the session (Fig. 4 F). We note this preliminary analysis on a single subject isn’t intended to make a claim about specific neurophysiological responses to emotional content in this brain region, but generally highlights that p-dLDS identifies meaningful dynamical modes where previous models are unable to.
Conclusion
In this work, we present a probabilistic decomposed linear dynamical systems model that can be used to discover meaningful representations in neural signals. By marginalizing over uncertainty in latent variable estimates and incorporating an offset into the dynamics, we enhance robustness and improve a variety of performance metrics.
Some areas of future work includes exploiting structure in the offsets to automatically identify window size and extending the probabilistic model to include more complicated emissions distributions, such as the Poisson likelihood commonly used to model
neural spiking data [ 24 ] .
A.1 p-dLDS Algorithm
Algorithm 1 describes the proposed inference algorithm. In our experiments, we set n=1 and \eta=10^{-4} and observe that the model converges. Below we use the notation hat notation for latent variable estimates or samples and the variable itself to represent the parameters of the variational distributions.
B.1 Lemma 1 Derivation
See 1

Let \bm{r}_{t}=\bm{x}_{t+1}-\bm{x}_{t} . We can rewrite the reconstruction objective in the following form,

\operatorname*{arg\,min}_{\bm{F}_{t},\bm{b}_{t}}\|\bm{r}_{t}-\bm{F}_{t}\bm{x}_{t}-\bm{b}_{t}\|_{2}^{2}+\lambda\|\bm{F}_{t}\|_{2}^{2}.

This objective is identical to the standard ridge regression with an unpenalized intercept term [ 12 ] . The solution is obtained by first centering the data, and then solving for the parameters using the solution for the standard Tikhonov regression. Below, we define the centered data as \bm{\tilde{x}}_{t} and \bm{\tilde{r}_{t}} for inputs and outputs respectively. Finally, we can use these values to obtain the following estimates of the parameters,

\bm{\widehat{b}}_{t}=\bm{\mu}_{t}\qquad\widehat{\bm{F}}_{t}=(\bm{\tilde{x}}_{t}^{\top}\bm{\tilde{x}}_{t}+\lambda\bm{I})^{-1}\bm{\tilde{x}}_{t}^{\top}\bm{\tilde{r}}_{t}

However, when there is only a single datapoint, we get that \tilde{\bm{x}}_{t}=0 , which results in \widehat{\bm{F}}=0 .
∎

This result arises from having only a single observation for any dynamic transition, which leads to a singular design matrix. Although we can improve our estimate of \bm{F}_{t} by collecting more samples along a given trajectory, this is impractical when dealing with naturalistic time-series. For instance, it may be infeasible to collect more data from the exact same initial condition in a naturalistic environment due to noise in the experimental setup. In chaotic systems, minor deviations can lead to drastically different outcomes over long time horizons. Even if it were possible to precisely control for the initial condition of the signal, the presence of dynamical noise can cause initially aligned time series to quickly drift out of alignment. Consequently, it is not uncommon to observe a single transition between any two time points, as it is not guaranteed that events across multiple trials will be well-aligned.
B.2 Lemma 2 Derivation
See 2

Case 1. Let \bm{l}\sim\mathcal{N}(\bm{\mu}_{l},\bm{\Sigma}_{l}) and \bm{b}\sim\mathcal{N}(\bm{\mu}_{b},\bm{\Sigma}_{b}) . The sum of normal random variables follows a distribution that results from convolving their individual distributions,

 q(\bm{x}) =q(\bm{l}+\bm{b}) =q(\bm{l})\ast q(\bm{b}) =\mathcal{N}(\bm{\mu}_{l},\bm{\Sigma}_{l})\ast\mathcal{N}(\bm{\mu}_{l},\bm{\Sigma}_{b}) =\mathcal{N}(\bm{\mu}_{l}+\bm{\mu}_{b},\bm{\Sigma}_{l}+\bm{\Sigma}_{b})

This is a standard result from probability theory.

Case 2. Now let \bm{l}\sim\mathcal{N}(\bm{\mu}_{l},\bm{\Sigma}_{l}+\bm{\Sigma}_{b}) and \bm{b}\sim\delta(\bm{b}-\bm{\mu}_{b}) . Similarly, the distribution of the sum of these variables is distributed according to their convolution,

 q(\bm{x}) =q(\bm{l})\ast q(\bm{b}) =\mathcal{N}(\bm{\mu}_{l},\bm{\Sigma}_{l}+\bm{\Sigma}_{b})\ast\delta(\bm{b}-\bm{\mu}_{b}) =\int_{-\infty}^{\infty}\mathcal{N}(\bm{x}-\bm{\tau};\bm{\mu}_{l},\bm{\Sigma}_{l}+\bm{\Sigma}_{b})\delta(\bm{\tau}-\bm{\mu}_{b})d\bm{\tau} =\mathcal{N}(\bm{x}+\bm{\mu}_{b};\bm{\mu}_{l},\bm{\Sigma}_{l}+\bm{\Sigma}_{b}) =\mathcal{N}(\bm{x};\bm{\mu}_{l}+\bm{\mu}_{b},\bm{\Sigma}_{l}+\bm{\Sigma}_{b}),

where the fourth line is the result of the sifting property of delta distributions. Since the final distribution in Case 1 and Case 2 are identical, we complete the proof.
∎
B.3 Optimal q(x) Update
The optimal coordinate ascent variational update is given by the following equation,

\begin{split}\log q^{*}(\bm{x})&\propto\mathbb{E}_{q(\bm{c},\bm{\gamma})}[\log p(\bm{x},\bm{c},\bm{y},\bm{\gamma}|\theta)]\ &=\mathbb{E}_{q(\bm{c},\bm{\gamma})}[\log p(\bm{l}_{1}|\theta)+\sum_{t=2}^{T}\log p(\bm{l}_{t}|\bm{l}_{t-1},\bm{c}_{t},\theta)+\sum_{t=1}^{T}\log p(\bm{y}_{t}|\bm{l}_{t}+\bm{b}_{t},\theta)]+C.\end{split} (11)

Conditioned on estimates of \bm{b}_{1:T} and samples of \bm{c}_{1:T} , the factor graph of equation ( 11 ) corresponds exactly to a time-varing Linear Gaussian State Space Model. Thus we can leverage the efficient inference algorithms such as the Kalman filter and RTS smoother when computing the marginals of the variational distribution of \bm{l}_{1:T} .
C.1 Noisy NASCAR
NASCAR data is generated by partitioning the two-dimensional state space into four regions according to the rules,

 Z(\bm{x})=\begin{cases}1,&x_{1}>1\ 2,&x_{1}<-1\ 3,&-1\leq x_{1}\leq 1,x_{2}\geq 0\ 4,&-1\leq x_{1}\leq 1,x_{2}<0,\end{cases}

where Z(\bm{x}) is the ground truth switching state function that depends on the particular location \bm{x} . The ground truth dynamics matrices are defined as,

\bm{A}(\bm{x})=\begin{cases}\begin{bmatrix}0&0.1\ -0.1&0\end{bmatrix},&\textrm{when }Z(\bm{x})=1\textrm{ or }2\[10.0pt] \begin{bmatrix}0&0\ 0&0\end{bmatrix},&\textrm{when }Z(\bm{x})=3\textrm{ or }4,\end{cases}

and ground truth offsets are defined as,

\bm{b}(\bm{x})=\begin{cases}\begin{bmatrix}0&0.005\end{bmatrix}^{\top},&\textrm{when }Z(\bm{x})=1\[5.0pt] \begin{bmatrix}0&-0.005\end{bmatrix}^{\top},&\textrm{when }Z(\bm{x})=2\[5.0pt] \begin{bmatrix}0.1&0\end{bmatrix}^{\top},&\textrm{when }Z(\bm{x})=3\[5.0pt] \begin{bmatrix}-0.1&0\end{bmatrix}^{\top},&\textrm{when }Z(\bm{x})=4.\end{cases}

Given the current location in state space \bm{x}_{t} , we can transition to the next point using the continuous time dynamics equation

\bm{x}_{t}={\rm expm}(\tau\bm{A}_{Z(\bm{x}_{t})})\bm{x}_{t-1}+\tau\bm{b}_{Z(\bm{x}_{t})}+\bm{ u}_{t},

where each entry of the process noise is sampled from \nu_{t,i}\sim\mathcal{N}(0,10^{-4}) . To modulate the speed of the system, we uniformly sample a speed constant \tau\in[0.1,1] , which is applied throughout each segment of the track. We use the continuous time formulation over the discrete-time formulation to ensure that changes to the speed do not distort the shape of the original system’s state space. To generate noisy observations, we construct a linear emissions matrix with random variables such that each entry is given by D_{i,j}\sim\mathcal{N}(0,1) .
C.2 Ramping Lorenz
In order to modulate the speed of the Lorenz system, we adjust the evaluation time points of an ODE integrator, specifically Runge-Kutta of the order 5(4) (RK54) as implemented in scipy’s solve_ivp [ 8 ] . Ramping activity is generated randomly with the following procedure,

Uniformly sample an evaluation interval length \tau\in[0.25,1.5] .

Construct a vector \widetilde{T} that consists n evenly spaced numbers over the interval [0,\tau] . In our experiments, we set n to be 100.

Perform the transformation \exp(\widetilde{T})-1 to obtain a vector of ramped evaluation times.

Plug in the transformed evaluation times into the RK45 Solver to obtain latent trajectories.

Similar to the NASCAR experiment, we generate noisy observation from a randomly constructed linear emissions matrix such that each entry is given by D_{i,j}\sim\mathcal{N}(0,1) .
C.3 Simulated Monkey Reaching Task
Our dataset is constructed from publicly available data and code from the center-out reach task in [ 19 , 7 ] . We obtain latent factors from spiking networks that are trained to reproduce empirically measured EMG signals, given a 3-dimensional input that specifies the go input and the reach angle. In our experiments, these factors are considered ground truth. Our trained factor-based spiking network then generates spiking activity for 1200 neurons. Synaptic currents are used as inputs into the Weighted Sum of synaptic currents LFP proxy method (WSLFP) [ 25 ] , as implemented in the wslfp Python package [ 15 , 14 ] . As WSLFP is a function of the relative location of neurons and electrodes, we place neurons randomly within a 5 mm by 10 mm by 1 mm region and electrodes in a grid centered in this region. The result is a multi-channel LFP dataset with nonlinear dynamics and measurements characteristic of systems neuroscience.
D.1 Multi-step Inference
The multi-step inference performance is computed with the following R-squared metric,

 R_{k}^{2}=1-\frac{\sum_{t=0}^{T-k}\|\bm{y}_{t+k}-\bm{\widehat{y}}_{t+k}\|_{2}^{2}}{\sum_{t=0}^{T-k}\|\bm{y}_{t+k}-\bm{\bar{y}}\|_{2}^{2}}, (12)

where k is the number of steps from the initial condition, \bm{\bar{y}} is the mean estimator for each trajectory and \bm{\widehat{y}}_{t+k} is the model prediction after applying the inferred dynamics for k steps. When testing, model parameters such as the dynamics and observation matrices are frozen, while specific latent variables are estimated based on the held-out data. In Table 1 , we show results for k=100 .
D.2 Inferred Dynamics Error
We measure the accuracy of the latent dynamics with the mean squared error (MSE) of the inferred speed, defined as,

{\rm MSE}_{\rm speed}=\frac{1}{T-1}\sum_{t=1}^{T-1}\|\bm{\dot{x}_{t}}-\bm{U}\bm{\widehat{\dot{x}}}_{t}\|_{2}^{2}, (13)

where the true speed \bm{\dot{x}}_{t}=\bm{x}_{t+1}-\bm{x}_{t} is computed from the denoised ground truth latent state, and the predicted speed \bm{\widehat{\dot{x}}}_{t}=\bm{\widehat{x}}_{t+1}-\bm{\widehat{x}}_{t} is computed using the model’s 1-step prediction. Since latent trajectories are only identifiable up to a linear transformation, we align the inferred trajectories with the true trajectories using a least squares fit before computing this score. More specifically, we find the optimal linear transformation \bm{U}\in\mathbb{R}^{N\times N} between the estimated and true states across all trajectories by solving,

\bm{\widehat{U}}=\operatorname*{arg\,min}_{\bm{U}}\frac{1}{T}\sum_{t=1}^{T}\|\bm{x}_{t}-\bm{U}\bm{\widehat{x}}_{t}\|
D.3 Inferred Latent State Space Error
Similarly, we measure the accuracy of the latent state space by computing the MSE after a linear alignment between trajectories from the inferred and true state space. We use this metric only for the reaching example, since the true observation function is a complex nonlinear function,

{\rm MSE}_{\rm state}=\frac{1}{T}\sum_{t=1}^{T}\|\bm{x_{t}}-\bm{U}\bm{\widehat{x}}_{t}\|_{2}^{2}

The linear alignment \bm{U}\in\mathbb{R}^{N\times N} between the estimated and true states across all trajectories is computed by solving the least squares problem in equation ( 14 ).
D.4 Inferred switching rate error
Evaluating the accuracy of the switching behavior is a more difficult task. In fact, developing a procedure that matches predicted switch times with true switch times can lead to a complicated optimal transport procedure. To simplify the evaluation of switching times, we marginalize over time, and compare only the MSE of the switch rate defined as,

{\rm MSE}_{\rm switch}=\frac{1}{m}\sum_{i=1}^{m}\|r_{i}-\widehat{r}_{i}\|_{2}^{2}, (16)

where m is the number of trials, r_{i} is the true switch rate for the i th trajectory, and \widehat{r}_{i}=\frac{1}{T}\sum_{t=1}^{T}{\bf 1}\{z_{t}\neq z_{t-1}\} is the predicted switch rate. Intuitively, \widehat{r}_{i} is the number of times that the state or dominant DO changes between consecutive time points normalized by the length of the interval T . In switching models, switch events are defined as a time point where the current inferred dynamical state differs from the state in the previous time step. Similarly in decomposed models, switch events are defined as time points where the active set of DOs change from the previous time step.

In the NASCAR example, r_{i} is defined with the number of transitions between ground truth segments. In the Lorenz example, r_{i} is defined by the number of times that the trajectory switches between the two lobes in addition to the number of ramping periods.
D.5 Reaching Classification Accuracy
We quantitatively evaluate the reaching experiment with a classification task. Here, we want to determine whether the learned systems can be used to distinguish between different reach directions. Recall that switched models infer a switching variable for each time point where z_{t}\in\{1,\dots,K\} while decomposed models infer a coefficient vector \bm{c}_{t}\in\mathbb{R}^{K} . Rather than viewing z_{t} as an index, we can equivalently view it as a one-hot encoded vector z_{t}\in\{0,1\}^{K} which describes whether a particular switching state is active at any given time. This matches the dimensionality of the variables in both switched and decomposed systems.

For simplicity, we focus on linear logistic regression classifiers in our experiment. If we let the inputs be z_{t} and c_{t} directly, then our classifiers quickly overfits since there are many more input features than trials. Specifically, the number of features scales linearly with the number of time points and systems \mathcal{O}(TK) . Instead, we marginalize over time and compute features from the estimated latent variables by averaging state activity over time. In switched models, this is the average one-hot encoding value over time. Similarly, this is the average coefficient value in decomposed models. However, for each dynamical state, we compute separate features for positive and negative coefficient values to prevent interference between them. In this setup, the input (feature) dimensionality scales according to \mathcal{O}(K) while the output dimensionality of the linear classifiers are the reaching directions. For all classifiers, we perform a grid search over the values \{10^{i}\}_{i=-4}^{4} to identify an appropriate amount of L2 regularization. Top-k accuracies are a standard metric in machine learning [ 18 , 3 ] and computed using the estimated class probabilities from the logistic regression classifier.
E.1 Synthetic Dynamical Systems
Figure 6 A demonstrates that our inference procedure converges to a local optimium while Figure 6 B shows a full sweep of the multi-step inference metric. Tables 1 in the main paper reports the final value. For completeness, we include Tables 3 and 4 which reports the means across 5 seeds of each model, and includes the standard deviations in parenthesis.