Title: Spectral Learning of Shared Dynamics Between Generalized-Linear Processes
Venue: NeurIPS 2024
Accepted: yes
Keywords: state space models

subspace identification

dynamical systems

neural coding

generalized-linear models
Abstract: Generalized-linear dynamical models (GLDMs) remain a widely-used framework within neuroscience for modeling time-series data, such as neural spiking activity or categorical decision outcomes. Whereas the standard usage of GLDMs is to model a single data source, certain applications require jointly modeling two generalized-linear time-series sources while also dissociating their shared and private dynamics. Most existing GLDM variants and their associated learning algorithms do not support this capability. Here we address this challenge by developing a multi-step analytical subspace identification algorithm for learning a GLDM that explicitly models shared vs. private dynamics within two generalized-linear time-series. In simulations, we demonstrate our algorithm's ability to dissociate and model the dynamics within two time-series sources while being agnostic to their respective observation distributions. In neural data, we consider two specific applications of our algorithm for modeling discrete population spiking activity with respect to a secondary time-series. In both synthetic and real data, GLDMs learned with our algorithm more accurately decoded one time-series from the other using lower-dimensional latent states, as compared to models identified using existing GLDM learning algorithms.

Spectral Learning of Shared Dynamics Between Generalized-Linear Processes

Abstract

Generalized-linear dynamical models (GLDMs) remain a widely-used framework within neuroscience for modeling time-series data, such as neural spiking activity or categorical decision outcomes. Whereas the standard usage of GLDMs is to model a single data source, certain applications require jointly modeling two generalizedlinear time-series sources while also dissociating their shared and private dynamics. Most existing GLDM variants and their associated learning algorithms do not support this capability. Here we address this challenge by developing a multi-step analytical subspace identification algorithm for learning a GLDM that explicitly models shared vs. private dynamics within two generalized-linear time-series. In simulations, we demonstrate our algorithm's ability to dissociate and model the dynamics within two time-series sources while being agnostic to their respective observation distributions. In neural data, we consider two specific applications of our algorithm for modeling discrete population spiking activity with respect to a secondary time-series. In both synthetic and real data, GLDMs learned with our algorithm more accurately decoded one time-series from the other using lowerdimensional latent states, as compared to models identified using existing GLDM learning algorithms.

1 Introduction

Generalized-linear dynamical models (GLDMs) are a commonly used framework for modeling dynamics using a low-dimensional latent variable that evolves over time [1-3]. Due to their interpretability, data efficiency, and amenability to real-time engineering operations, GLDMs remain a widely popular tool in neuroscience for modeling time-series data, whether it be Poisson spiking neural activity, Bernoulli/Binomial categorical task variables, or Gaussian behavior [4-9]. Whereas most existing GLDM variants and their associated analytical learning algorithms focus on modeling the dynamics within a single time-series, there exist applications that require explicit dissociation of shared vs. private dynamics within two generalized-linear observation sources. For example, such functionality is helpful when modeling the dynamical relationship between recorded neural activity and certain behaviors of interest [10-15].

Here we fill these methodological gaps by deriving a novel covariance-based subspace system identification (SSID) algorithm that is capable, with its multi-staged learning approach, of identifying the shared dynamics between two generalized-linear time-series with priority, before modeling the dynamics private to each observation. We design the method to seamlessly generalize to different observation distributions, such as Poisson or Bernoulli. To illustrate the method, we first show in simulations that our method successfully dissociates the shared dynamics within two generalizedlinear time-series, agnostic of their respective observation models; to compare against existing GLDM methods, we focused on Poisson, Bernoulli, and Gaussian generalized-linear observations. Next we demonstrate our method on two public non-human primate (NHP) datasets of discrete population spiking activity recorded from different brain regions and during different contexts [1618]. Compared with existing Poisson GLDMs and their learning algorithms, our method learned models that more accurately decoded one time-series from the other using lower-dimensional latent states, suggesting improved learning of shared dynamics.

2 Background

For linear state-space models with continuous Gaussian observations, subspace system identification (SSID) theory provides computationally efficient non-iterative algorithms for analytically learning state-space models, both with and without identification of shared dynamics [14, 19-24]. These methods, however, either are not applicable to generalized-linear time-series with non-Gaussian observations [14, 19, 20, 23] or do not have the ability to dissociate shared vs. private dynamics between two time-series [21, 22, 24]. To help with the exposition of our method in section 3, we first review standard covariance-based SSID and an existing SSID method for modeling Poisson point-processes [21], a widely-used class of generalized-linear observations.

2.1 Standard covariance-based SSID

The standard formulation for a linear state-space model with continuous Gaussian observations is as

where x k ∈ R n x is the latent state variable, r k ∈ R n r corresponds to continuous Gaussian observations, and w and v are state and observation noise terms, respectively, with distributions N ( w k ; 0 , Q ) and N ( v k ; 0 , R ) , and cross-covariance S . Further, we define G := Cov( x k +1 , r k ) , as the covariance between future latent state and current observation, and Λ r 0 := Cov( r k , r k ) , as the instantaneous covariance of the observations. Standard covariance-based SSID learns the parameters of a latent dynamical system Θ = ( A , C r , G , Λ r 0 ) given training samples r k and hyperparameter n x that specifies the latent state dimensionality. To do so, a future-past Hankel matrix, H r , is first constructed from the cross-covariances of the system's linear observations as [19, 20]

where the integer i denotes the user-specified maximum temporal lag (i.e., horizon) used to construct H r and Λ r τ := Cov( r k + τ , r k ) is the τ -th lag cross-covariance for any timepoint k , under timestationary assumptions. We note that the rank of H r must be at least n x in order to identify a

model with a latent dimension of n x . Thus the user-specified horizon i must satisfy i × n r ≥ n x . Covariance-based SSID then decomposes H r into a product of observability ( Γ r ) and controllability ( ∆ ) matrices as [19, 20]

where G is defined as above. The factorization of H r is done by computing a singular value decomposition (SVD) of H r and keeping the top n x singular values and corresponding singular vectors. From the factors of H r , C r is read off as the first n r rows of Γ r and G is read off as the last n r columns of ∆ . A is learned by solving Γ r = Γ r A , where Γ r and Γ r denote Γ r from which the top or bottom n r rows have been removed, respectively. This optimization problem has the following closed-form least-squares solution A = Γ † r Γ r , with † denoting the pseudo-inverse operation. The final parameter Λ r 0 is computed as the empirical covariance of r k . See appendix A.4 on how ( G , Λ r 0 ) specify ( Q , R , S ) .

2.2 SSID for a single generalized-linear time-series

There has been some work extending SSID to generalized-linear time-series, such as Poisson and Bernoulli observations [21, 24]. These methods, however, only learn the dynamics of a single generalized-linear time-series rather than model shared vs. private dynamics between two time-series. Here we present one of our baselines, PLDSID [21], which models a single Poisson time-series, as an example. A Poisson linear dynamical system (PLDS) model is defined as

where x k ∈ R n x is the latent state as before and y k ∈ R n y corresponds to discrete (e.g., neural spiking) observations which, conditioned on the latent process r k , are Poisson-distributed with a rate equal to the exponential of r k (i.e., log-rate). Finally, w k is Gaussian-distributed state noise with covariance parameter Q , as before, and b is a constant baseline log-rate. The PLDS model is commonly used for modeling Poisson process events, such as neural spiking activity [2, 4, 6, 21, 25]. Buesing et al. [21] developed a SSID algorithm, termed PLDSID, to learn the PLDS model parameters Θ PLDS = ( A , C r , b , Q ) given training samples y k and hyperparameter n x .

Standard covariance-based SSID algorithms (section 2.1) are not directly applicable to Poissondistributed observations. This is because the log-rates r k that are linearly related to the latent states in equation (4) are not observable in practice - rather, only a stochastic Poisson emission from them (i.e., y k ) is observed. As a result, the second moments constituting H r (i.e., Λ r τ ) cannot be directly estimated. The critical insight by Buesing et al. [21] was to leverage the log link function (i.e., exp -1 ) and the known conditional distribution y k | r k to compute the first ( µ r ± ) and second ( Λ r ± ) moments of the log-rate r k from the first ( µ y ± ) and second ( Λ y ± ) moments of the discrete observations y k . The ± denotes that moments are computed for the future-past stacked vector of observations r ± := [ r T f r T p ] T and y ± := [ y T f y T p ] T , where

To compute moments of the log-rate, Buesing et al. [21] derived the following moment conversion

̸

where m = n correspond to different indices of the first and second moments of the future-past stacked observation vectors r ± and y ± , and n, m = 1 , · · · , Kn y where K is the total number of time points. With the first and second moments computed in the moment conversion above, the baseline log rate b parameter is read off the first n r rows of µ r ± and the Hankel matrix, H r , is constructed as per equation (2). From here, we can proceed with the standard covariance-based SSID algorithm using H r , as outlined in section 2.1. Discussion regarding learning the state noise covariance parameters (e.g., Q ) is postponed to appendix section A.3, where we use an approach that - unlike Buesing et al. [21] - ensures validity of learned noise statistics.

3 Method

3.1 Model definition and assumptions

Both the standard linear state-space model (equation (1)) and PLDS model (equation (4)) only model a single observation on its own. To enable identification of shared and private dynamics between two generalized-linear time-series, we write the following general multi-observation GLDM

where x k , r k , w k , v k , and b are defined as in equations (1) and (4). We introduce t k ∈ R n t to represent the second generalized-linear observation time-series, z k ∈ R n z (with n t = n z by construction) to represent the latent process underlying this second observation, ϵ k ∼ N ( ϵ k ; 0 , F ) to represent the associated noise term, and d to represent the associated baseline value. We generically denote the probability distribution for y k conditioned on the latent r k with P y . For example, in the PLDS model P y | r := Poisson(exp( r k )) . P t | z is defined similarly but for t k and z k . Finally, g ( · ) and h ( · ) correspond to the link function in the generalized-linear model, for example g ( r k ) = exp( r k ) in PLDS models. In order to dissociate between shared and private dynamics within the observation time-series, we introduce the following definition:

Definition 3.1. We take the system to be written in a block structure form as defined below [14], allowing us to dissociate shared from private latents

where x (1) k ∈ R n 1 corresponds to latent states that drive both z k and r k , x (2) k ∈ R n 2 corresponds to states that only drive r k , and x (3) k ∈ R n 3 corresponds to states that only drive z k - with total states n x = n 1 + n 2 + n 3 . The parameter G can also be written in block partition format such that

We can further simplify the definition of G using the following assumptions:

Assumption 3.2. The state noise covariance Q is assumed to have a block diagonal structure such that Q = diag( Q (1 , 2) , Q (3) ) , where Q (1 , 2) is a square matrix of dimension n 1 + n 2 and Q (3) is a square matrix of dimension n 3 . Formally, the superscript notation (1 , 2) designates attribution of the parameter to the first and second set of latent states.

Assumption 3.3. Initial latent states are assumed to be mutually-independent, making Cov( x 0 , x 0 ) diagonal.

These assumptions allow us to fully decouple the private latent states of the secondary time-series ( x (3) ) from the latent states driving the primary time-series ( x (1) , x (2) ). As a result, we can take G (3) = Cov( x (3) k +1 , r k ) = 0 . From the perspective of state estimation, this simplification implies that r k provides no information to help estimate x (3) k +1 , for all k ; this understanding is consistent with our definition of shared and private states.

3.2 Prioritized generalized-linear dynamical modeling (PGLDM)

Our method, which we term Prioritized Generalized-Linear Dynamical Modeling (PGLDM), uses a multi-staged learning approach to model a primary generalized-linear time-series while prioritizing identification of the dynamics shared with a secondary time-series. Note, 'primary" refers to the data source whose modeling is of primary interest and that can optionally be used to predict the

secondary data source. For example, within the context of decoding continuous behaviors from discrete population spiking activity, Poisson observations (i.e., neural activity) are the primary timeseries whereas Gaussian observations (e.g., kinematics) are the secondary time-series. During stage 1, shared dynamics are learned using both observations. In stage 2, any private dynamics in the primary time-series are optionally learned. This two-staged approach allows prioritized learning of shared dynamics in the sense that latent states will be dedicated to explaining non-shared dynamics in the primary time-series only if there are enough latent states to explain the shared dynamics. Finally, an optional stage 3 allows identification of the dynamics private to the secondary time-series.

Below we will outline the first two stages of our algorithm and leave the optional stage 3 to appendices A.1.4 and A.9. A crucial component of our method is a new covariance-based SSID algorithm for identifying shared and private dynamics between two observation time-series using their first and second moments only. This covariance-based algorithm is what enables our method to be applicable to generalized-linear observations, which we will expand on further in section 3.2.3. We first present this new covariance-based SSID algorithm, or equivalently PGLDM for linear state-space models with continuous Gaussian observations (the first three lines of equation (6)), before showing support for generalized-linear time-series broadly. The derivation of PGLDM is provided in appendix A.1.

3.2.1 Stage 1: shared dynamics

In the first stage, our algorithm identifies the parameter set corresponding to the shared dynamical subspace, ( A 11 , C (1) r , C (1) z , b , d ) , given hyperparameter n 1 and using both z k and r k , both of which are observable in the Gaussian case (i.e., equation (1) or equivalently the first three lines of equation (6)). To do this, we first construct a Hankel matrix between future observations of the secondary process and past observations of the primary process

with z f := [ z i . . . z 2 i -1 ] T and r p defined as in equation (2). Although equation (8) uses the same horizon for both observations, in practice we implement the method for a more general version with distinct horizon values i r for the primary observations and i z for the secondary observations, resulting in H zr ∈ R i z ∗ n z × i r ∗ n r . This allows users to independently specify the horizons for the two observations, which can improve modeling accuracy especially if the two observations have very different dimensionalities (see section 4.2 and appendix A.1.6). After constructing H zr , we decompose it using SVD and keep the top n 1 singular values and their corresponding singular vectors

where n 1 is the user-specified dimensionality of the shared latent states x (1) k , Γ z denotes the observability matrix for the secondary observations, and ∆ (1) denotes the controllability matrix associated with the shared latent states (defined as in equations (3) and (16) in the appendix). At this point, we extract C (1) z by reading off the first n z rows of Γ z . To extract C (1) r we first form H r per equation (2) and extract the observability matrix for r associated with the shared latent dynamics, Γ (1) r , by right multiplying H r with the pseudoinverse of ∆ (1)

We then read C (1) r from the first n r lines of Γ (1) r (defined as in equation (20) in the appendix). The baseline parameters b and d are empirically computed as the means of r k and z k , respectively.

Lastly, to learn the shared dynamics summarized by the parameter A 11 , we solve the optimization problem ∆ (1) = A 11 ∆ (1) where ∆ (1) and ∆ (1) denote ∆ (1) from which n r columns have been removed from the right or left, respectively. The closed-form least-squares solution for this problem is A 11 = ∆ (1) ( ∆ (1) ) † . This concludes the learning of the desired parameters ( A 11 , C (1) r , C (1) z , b , d ) , given hyperparameter n 1 .

3.2.2 Stage 2: private dynamics in primary process

After learning the shared dynamics, our algorithm can learn the dynamics private to the primary process that were not captured by x (1) k . Specifically, we learn the following parameters from equation (7): ([ A 21 A 22 ] , C (2) r ) , with hyperparameter n 2 determining the unshared latent dimensionality of r . To do so, we first compute a 'residual' Hankel matrix, H (2) r , using Γ (1) r and ∆ (1) from stage 1 and decompose it using SVD, keeping the first n 2 singular values and vectors

With C (2) r , which corresponds to the first n r rows of Γ (2) r , we construct C r = [ C (1) r C (2) r ] . We then use ∆ (2) to form the controllability matrix ∆ (1 , 2) as the concatenation of ∆ (1) and ∆ (2) (derivation in appendix A.1):

where A (1 , 2) refers to the upper left block of the dynamics matrix A that corresponds to the latent states x (1) and x (2) . Given ∆ (1 , 2) , we extract [ A 21 A 22 ] by solving the problem ∆ (2) = [ A 21 A 22 ] ∆ (1 , 2) where

Concatenating the sub-blocks together, A (1 , 2) = [ A 11 0 A 21 A 22 ] . Thus, given hyperparameters n 1

and n 2 , we now have all model parameters associated with the shared dynamics and dynamics private to the primary signal: ( A (1 , 2) , C r , C (1) z , b , d ) . The remaining model parameters ( A 33 , C (3) z ) are learned in stage 3 (appendices A.1.4 and A.9).

3.2.3 Supporting generalized-linear processes

The covariance-based SSID algorithm that we have just derived is what enables our framework to be broadly applicable to generalized-linear time-series data. As discussed in section 2.2, the variables that are linearly related to latent states x are unobservable in generalized-linear models. However, because the algorithm outlined in sections 3.2.1-3.2.2 only relies on empirical covariances and cross-covariances of the two observation time-series, we can support generalized-linear processes by using moment-conversions (e.g., section 2.2) [21, 24]. When computationally tractable moment conversion equations exist, we can compute both a Hankel matrix H r , as described in section 2.2, and also a cross-term Hankel matrix H zr . For example, for the scenario wherein the Poisson observations constitute the primary process and Gaussian observations the secondary process (i.e., the first four lines of equation (6)), we can compute a moment conversion to estimate joint moments of z k and r k from the joint moments of the observed signals z k and y k with the following equation (derived using the conditional statistical properties, see appendix A.1.5)

where, similar to equation (5), m and n correspond to indices of the first and second moments of the observation vectors z f and r p (or y p ), respectively. As another example, if both generalized-linear observations (e.g., y and t from equation (6)) are Poisson distributed, the joint moments can be computed as

For both of these scenarios the baseline log-rates are learned as in PLDSID (section 2.2). Thus, our novel multi-staged covariance-based SSID learning algorithm enables identification of shared vs. private dynamics across various generalized-linear processes.

4 Experimental Results

4.1 Shared dynamics are accurately identified in generalized-linear simulations

To evaluate how well our method identified the shared dynamics between two generalized-linear time-series, we simulated observations from random dynamical models as per equation (6). All state and observation dimensions were randomly selected, and the corresponding system parameters were randomly generated to simulate stable and slow-decaying dynamics (see appendix A.7.1 for details). In our first experiment, we evaluated how well the shared dynamical subspace could be identified when learning models at the true shared dimensionality. We performed this analysis for four combinations of generalized-linear observation pairs ( r k or y k and z k or t k , respectively): (1) Gaussian/Gaussian, (2) Poisson/Gaussian, (3) Poisson/Poisson, and (4) Bernoulli/Gaussian. Within each configuration, we compared models learned by our method against models learned with either Laplace-EM (expectation-maximization) [26] or a SSID algorithm with the appropriate observation distribution. For the Gaussian/Gaussian case we also compare against PSID [14], an SSID algorithm that preferentially learns the shared dynamics between two Gaussian time-series. PGLDM (our method) and PSID were trained using both the primary and secondary time-series, whereas all other methods used only the primary time-series as they only model a single data source. We evaluated identification of shared dynamics by computing the normalized eigenvalue error between ground truth shared modes (i.e., eigenvalues of A 11 in equation (7)) and the identified modes (i.e., the learned A 11 for PGLDM/PSID or A for the other baselines); see appendix A.8.1 for evaluation details. We report the results of this analysis for 20 systems per configuration in Table 1. For almost all conditions PGLDM more accurately identified the shared dynamics.

In our second simulation experiment, we studied the effect of latent state dimension on learning. We generated 16 systems with fixed dimensions for shared and private latent states given by n 1 = 4 , n 2 = 12 , and n 3 = 4 , accordingly. We swept the learned latent state dimension from 1 to the true dimensionality of the primary observation time-series n 1 + n 2 = 16 , with the dimensionality of shared dynamics set to min( current n x , n 1 ) . We found that our method accurately identified the shared modes with the minimal latent state dimension of 4; in contrast, PLDSID and Laplace-EM did not reach such high accuracy even when using higher latent state dimensions (figure 1c). In these simulations we also evaluated the predictive power (i.e., correlation coefficient, CC) of the model when using discrete Poisson observations to predict continuous Gaussian observations in a held-out test set (see appendix A.8.2). This second metric allowed us to test our hypothesis that PGLDM's explicit modeling of the shared subspace improved decoding of Gaussian observations from Poisson observations compared with our baselines PLDSID [21] and Laplace-EM [26]. We observed that our method achieved higher decoding performance in low-dimensional regimes, even when using as few as 4 latent states, whereas PLDSID required much larger latent state dimensions (around 12) to reach comparable performance (figure 1a). We also evaluated Poisson self-prediction using area under the receiver operating characteristic curve, AUC (figure 1b). With the inclusion of stage 2 and sufficient model capacity, models learned by PGLDM were able to achieve comparable performance in self-prediction as compared to our baselines.

(a)

(q)

0.78

Correlation Coefficient

FZ'O

Poisson Self-prediction

(CC)

0.5

Oracle model

performance

AUC

0.70

Oraclemodel

performance

True number of

sharedlatentstates

Truenumberof

0.3-

0.62

一

sharedlatentstates

2

4

6

8

10

12

14

16

2

4

6

8

10

12

14

16

Number of Latent States (nx)

Number of Latent States (nx)

(c)

PGLDM

(p)

Identification Error (log10)

PLDSID

-0.5

Normalized

-2.0

Sharedmodes

Y-onlymodes

双

Z-only modes

PGLDM

X

-3.5

bestLDS

2

T

4

6

8

T

10

12

14

16

×

Number of Latent States (nx)

4.2 Modeling shared dynamics improves motor decoding from population spiking activity

As a demonstration on real data, we used PGLDM to model the shared dynamics between discrete Poisson population neural spiking activity and continuous Gaussian arm movements in a publicly available NHP dataset from the Sabes lab [17]. The dataset is of a NHP moving a 2D-cursor in a virtual reality environment based on fingertip position. We use the 2D cursor position and velocity as the continuous observations z . For all methods we used 50ms binned multi-unit spike counts for the discrete observations y . We evaluated decoding performance of learned models using five-fold cross validation across six recording sessions (see appendix A.7.2 for cross-validation details). For PGLDM, we use the shared dynamics dimensionality of n 1 = min( current n x , 8) , i.e., a maximum n 1 of 8, because behavior decoding using stage 1 roughly plateaued at this dimension.

Compared with PLDSID and Laplace-EM, our method learned models that led to better behavioral decoding at all latent state dimensions, including at the maximum latent state dimension (figure 2a). This result suggests that our method better learns the shared dynamics between Poisson spiking and continuous movement observations due to its ability to dissociate shared vs. private latent states. Interestingly, despite the focus on learning the shared latent states in the first stage, PGLDM was also able to extract the private latent states in the Poisson observations because of its second stage. This stage led to improved neural self-prediction AUC, while still maintaining the more accurate behavioral decoding (figure 2b-c). Indeed, even with the inclusion of just two additional latent states to model private Poisson dynamics ( n 2 = 2 , n x = 10 ), neural self-prediction was approaching that of models learned by PLDSID (figure 2b). Finally, given its analytical nature, PGLDM required a substantially lower training time compared with Laplace-EM (see appendix Table 2).

(a)

(q)

0.45

AU

490

Kinematics prediction

I self-prediction

0.35-

0.65

PGLDM

0.25

F90

PLDSID

Poisson

0.15

Laplace-EM

Neural

0.61

0.59

2

4

6

8

10

12

2

4

6

8

10

12

Number of Latent States(nx)

Number of Latent States(nx)

(c)

Kinematics prediction

0.70

水水*

Neural self-prediction

(d)

***

0.70

Cursor

Cursor

x-position

y-position

CC

AUC

Cursor

Cursor

x-velocity

y-velocity

0.20

0.62

nx=12

nx=12

10seconds

4.3 PGLDM models better decode spiking activity of one visual area from another

As a second demonstration on real data but with a different combination of observation distributions (Poisson/Poisson), we used PGLDM to decode neural population spiking activity in one visual area from another. In a publicly available dataset from Zandvakili and Kohn [16, 18], simultaneous V1/V2 population recordings were performed in anaesthetized NHPs as they were presented visual stimuli. We used five-fold cross validation to evaluate learned model performance in decoding V1 activity from V2 activity and in V2 self-prediction. We again compare with PLDSID and LaplaceEM. For all learning algorithms we tested four latent state dimensions such that n x = n 1 + n 2 ∈ { 2 , 4 , 6 , 8 } . For PGLDM we used the first two stages, setting the shared dynamics dimensionality to n 1 = min( current n x , 4) . Similar to results in figure 2, we chose a maximum n 1 of 4 because decoding roughly plateaued at this dimension. The conclusions were consistent with those in figure 2: modeling the shared vs. private dynamics by PGLDM allowed for better decoding of V1 activity while maintaining comparable self-prediction of V2 activity. Analysis details are in appendix A.7.3. We present the results of the complementary analysis (i.e., predicting V1 from V2) in appendix A.12.

4.4 Limitations

PGLDM, similar to other SSID methods, uses a time-invariant model which may not be suitable if the data exhibits non-stationarity, such as in chronic neural recordings. To handle non-stationarities, one would need to either intermittently refit the model after a predetermined duration of time, or develop adaptive extensions [27, 28]. As an example of the latter, one can gradually update the model parameters by incorporating a learning rate that weighs recent observations more heavily in the moment computations while gradually forgetting past observations [27]. Moreover, as with other covariance-based SSID methods, PGLDM may be sensitive to the accuracy of the empirical estimates of the first- and second-order moments. However, with increasing number of samples these empirical estimates will approach true statistical values, thereby improving overall performance, as seen in appendix figure 4. Due to errors in the empirical estimates of the covariances, SSID methods may also occasionally learn unstable dynamics (see appendix A.5). Future work may address this by

(a)

(b)

AUC

AUC

0.60-

self-prediction

0.550

n1=4

0.58-

PGLDM

0.540

PLDSID

0.56-

Poisson

Laplace-EM

0.530

0.54

n1=4

0.520

2

4

6

8

2

4

6

8

Number ofLatentStates(nx)

NumberofLatentStates(nx)

(c)

(p)

0.80-

0.65

AUC

AUC

0.70

2self-prediction

0.60

decoding

0.60-

0.55

0.50-

0.50

V1

0.40

0.45

nx=8

nx=8

incorporating techniques from control theory, such as mode stabilization and covariance matching [2932]. Finally, although the GLDMs that PGLDM learns are widely used (e.g., in neuroscience), such models may not be suitable for time-series with nonlinearly evolving states. We did not compare our method against nonlinear deep learning methods, such as recurrent neural networks and transformers [33-38], because the goals of these two modeling approaches are different. While nonlinear deep learning methods are typically used to boost overall decoding performance, GLDMs are used for their interpretability and utility in scientific investigations and in real-time, computationally-efficient engineering applications (e.g., brain-computer interfaces).

5 Discussion

We developed PGLDM, a novel analytical multi-staged covariance-based SSID algorithm for modeling two generalized-linear processes while also dissociating shared from private dynamics. In simulations we demonstrate that our method successfully achieves this capability agnostic to the generalized-linear observation distribution. As a result, our approach more accurately models system dynamics compared to several commonly-used GLDM variants and their corresponding learning algorithms. We also demonstrate our method's applicability to real data by modeling two distinct NHP datasets recorded under different contexts and from different brain regions. In both simulations and in real data, PGLDM's ability to dissociate shared from private dynamics improved decoding of a secondary time-series from a primary time-series despite using lower-dimensional latent states. Further, although here we specifically focused on modeling Gaussian, Poisson, and Bernoulli observations, our algorithm can be extended to alternate distributions described with generalized-linear models or to other link functions than the ones used here, as long as there exists a corresponding computationally tractable moment conversion equation. This is possible, if a closed-form equation exists, because the covariance-based approach of PGLDM only requires the second-order moments after moment conversion (equations (2), (5), (8), (11), (12)); as such, in these scenarios the moment conversion algorithm can be modified for the desired link function and/or generalized-linear observation model [21, 24]. Beyond neuroscience, due to the high-prevalence of GLDMs across various application domains, our method may be a useful tool for modeling the shared and private dynamics of joint generalized-linear processes with distinct observation distributions.

Acknowledgments and Disclosure of Funding

This work was supported by Office of Naval Research (ONR) YIP grant N00014-19-1-2128, Army Research Office (ARO) MURI grant W911NF-16-1-0368, and National Institutes of Health (NIH) grants DP2MH126378, R01MH123770, and Brain Initiative R61MH135407. M.M.S. is an inventor on University of Southern California's patents or patent applications related to decoding and closedloop control approaches, and is a consultant for Paradromics Inc. Authors thank Han-Lin Hsieh, Christian Song, and Parima Ahmadipouranari for helpful discussions.

References

Liam Paninski, Yashar Ahmadian, Daniel Gil Ferreira, Shinsuke Koyama, Kamiar Rahnama Rad, Michael Vidne, Joshua Vogelstein, and Wei Wu. A new look at state-space models for neural data. Journal of computational neuroscience , 29(1-2):107-126, August 2010. ISSN 1573-6873 0929-5313. doi: 10.1007/s10827-009-0179-x.

J. H. Macke, L. Buesing, and M. Sahani. Estimating state and parameters in state space models of spike trains , page 137-159. Cambridge University Press, 2015. doi: 10.1017/CBO9781139941433.007.

Scott Linderman, Ryan P Adams, and Jonathan W Pillow. Bayesian latent structure discovery from multi-neuron recordings. In Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016.

Wilson Truccolo, Uri T. Eden, Matthew R. Fellows, John P. Donoghue, and Emery N. Brown. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. Journal of Neurophysiology , 93(2):1074-1089, 2005. doi: 10.1152/jn.00697.2004. PMID: 15356183.

Jonathan W. Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher, Alan M. Litke, E. J. Chichilnisky, and Eero P. Simoncelli. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature , 454(7207):995-999, August 2008. ISSN 1476-4687. doi: 10.1038/nature07140.

Vernon Lawhern, Wei Wu, Nicholas Hatsopoulos, and Liam Paninski. Population decoding of motor cortical activity using a generalized linear model with hidden states. Journal of Neuroscience Methods , 189(2):267-280, June 2010. ISSN 0165-0270. doi: 10.1016/j.jneumeth.2010.03.024.

Hung-Yun Lu, Elizabeth S Lorenc, Hanlin Zhu, Justin Kilmarx, James Sulzer, Chong Xie, Philippe N Tobler, Andrew J Watrous, Amy L Orsborn, Jarrod Lewis-Peacock, and Samantha R Santacruz. Multiscale neural decoding and analysis. Journal of Neural Engineering , 18(4):045013, Aug 2021. doi: 10.1088/1741-2552/ac160f.

Yuxiao Yang, Shaoyu Qiao, Omid G. Sani, J. Isaac Sedillo, Breonna Ferrentino, Bijan Pesaran, and Maryam M. Shanechi. Modelling and prediction of the dynamic responses of large-scale brain networks during direct electrical stimulation. Nature Biomedical Engineering , 5(4):324-345, April 2021. ISSN 2157-846X. doi: 10.1038/s41551-020-00666-w.

Edoardo Balzani, Jean-Paul G Noel, Pedro Herrero-Vidal, Dora E Angelaki, and Cristina Savin. A probabilistic framework for task-aligned intra- and inter-area neural manifold estimation. In The Eleventh International Conference on Learning Representations , 2023.

Jonathan C. Kao, Paul Nuyujukian, Stephen I. Ryu, Mark M. Churchland, John P. Cunningham, and Krishna V. Shenoy. Single-trial dynamics of motor cortex and their applications to brain-machine interfaces. Nature Communications , 6:7759, July 2015. ISSN 2041-1723. doi: 10.1038/ncomms8759.

Mehdi Aghagolzadeh and Wilson Truccolo. Inference and decoding of motor cortex low-dimensional dynamics via latent state-space models. IEEE Transactions on Neural Systems and Rehabilitation Engineering , 24(2):272-282, 2016. doi: 10.1109/TNSRE.2015.2470527.

William E. Allen, Michael Z. Chen, Nandini Pichamoorthy, Rebecca H. Tien, Marius Pachitariu, Liqun Luo, and Karl Deisseroth. Thirst regulates motivated behavior through modulation of brainwide neural population dynamics. Science , 364(6437):eaav3932, April 2019. doi: 10.1126/science.aav3932.

Carsen Stringer, Marius Pachitariu, Nicholas Steinmetz, Charu Bai Reddy, Matteo Carandini, and Kenneth D. Harris. Spontaneous behaviors drive multidimensional, brainwide activity. Science , 364(6437): eaav7893, April 2019. ISSN 0036-8075, 1095-9203. doi: 10.1126/science.aav7893.

Omid G. Sani, Hamidreza Abbaspourazad, Yan T. Wong, Bijan Pesaran, and Maryam M. Shanechi. Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification. Nature Neuroscience , 24(1):140-149, January 2021. ISSN 1546-1726. doi: 10.1038/s41593-020-00733-0.

Evren Gokcen, Anna I. Jasper, João D. Semedo, Amin Zandvakili, Adam Kohn, Christian K. Machens, and Byron M. Yu. Disentangling the flow of signals between populations of neurons. Nature Computational Science , 2(8):512-525, August 2022. ISSN 2662-8457. doi: 10.1038/s43588-022-00282-5.

Amin Zandvakili and Adam Kohn. Coordinated neuronal activity enhances corticocortical communication. Neuron , 87(4):827-839, August 2015. doi: 10.1016/j.neuron.2015.07.026.

Joseph E. O'Doherty, Mariana M. B. Cardoso, Joseph G. Makin, and Philip N. Sabes. Nonhuman Primate Reaching with Multichannel Sensorimotor Cortex Electrophysiology, May 2017. URL https: //doi.org/10.5281/zenodo.583331 .

Amin Zandvakili and Adam Kohn. Simultaneous V1-V2 neuronal population recordings in anesthetized macaque monkeys, 2019. URL https://crcns.org/data-sets/vc/v1v2-1 .

Peter Van Overschee and Bart De Moor. Subspace Identification for Linear Systems . Springer US, Boston, MA, 1996. ISBN 978-1-4613-8061-0. doi: 10.1007/978-1-4613-0465-4.

Tohru Katayama. Subspace Methods for System Identification . Springer London, 2005. doi: 10.1007/ 1-84628-158-x.

Lars Buesing, Jakob H Macke, and Maneesh Sahani. Spectral learning of linear dynamics from generalisedlinear observations with application to neural population data. Advances in Neural Information Processing Systems , 25:9, 2012.

Parima Ahmadipour, Omid G. Sani, Bijan Pesaran, and Maryam M. Shanechi. Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity. Journal of Neural Engineering , 2023.

Parsa Vahidi, Omid G Sani, and Maryam M Shanechi. Modeling and dissociation of intrinsic and inputdriven neural population dynamics underlying behavior. Proceedings of the National Academy of Sciences , 121(7):e2212887121, 2024.

Iris R Stone, Yotam Sagiv, Il Memming Park, and Jonathan W. Pillow. Spectral learning of bernoulli linear dynamical systems models for decision-making. Transactions on Machine Learning Research , 2023. ISSN 2835-8856.

Anne C. Smith and Emery N. Brown. Estimating a state-space model from point process observations. Neural Computation , 15(5):965-991, May 2003. ISSN 0899-7667. doi: 10.1162/089976603765202622.

Scott Linderman, Benjamin Antin, David Zoltowski, and Joshua Glaser. SSM: Bayesian Learning and Inference for State Space Models, October 2020. URL https://github.com/lindermanlab/ssm .

Parima Ahmadipour, Yuxiao Yang, Edward F. Chang, and Maryam M. Shanechi. Adaptive tracking of human ECoG network dynamics. Journal of Neural Engineering , 18:016011, 2 2021. ISSN 1741-2552. doi: 10.1088/1741-2552/ABAE42.

Yuxiao Yang, Parima Ahmadipour, and Maryam M. Shanechi. Adaptive latent state modeling of brain network dynamics with real-time learning rate optimization. Journal of Neural Engineering , 18(3):036013, March 2021. ISSN 1741-2552. doi: 10.1088/1741-2552/abcefd.

J. M. Maciejowski. Guaranteed stability with subspace methods. Systems & Control Letters , 26(2): 153-156, 1995. ISSN 0167-6911. doi: https://doi.org/10.1016/0167-6911(95)00010-7.

Anders Lindquist and Giorgio Picci. Canonical correlation analysis, approximate covariance extension, and identification of stationary time series. Automatica , 32(5):709-733, 1996. ISSN 0005-1098. doi: https://doi.org/10.1016/0005-1098(96)80649-2.

Christopher I. Byrnes, Sergei V. Gusev, and Anders Lindquist. A convex optimization approach to the rational covariance extension problem. SIAM Journal on Control and Optimization , 37(1):211-229, 1998. doi: 10.1137/S0363012997321553.

Brien Alkire and Lieven Vandenberghe. Convex optimization problems involving finite autocorrelation sequences. Mathematical Programming , 93(3):331-359, December 2002. ISSN 1436-4646. doi: 10.1007/ s10107-002-0334-x.

Chethan Pandarinath, Daniel J. O'Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D. Stavisky, Jonathan C. Kao, Eric M. Trautmann, Matthew T. Kaufman, Stephen I. Ryu, Leigh R. Hochberg, Jaimie M. Henderson, Krishna V. Shenoy, L. F. Abbott, and David Sussillo. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature Methods , 2018. doi: 10.1038/s41592-018-0109-9.

Cole Hurwitz, Akash Srivastava, Kai Xu, Justin Jude, Matthew Perich, Lee Miller, and Matthias Hennig. Targeted neural dynamical modeling. In Advances in Neural Information Processing Systems , volume 34, pages 29379-29392. Curran Associates, Inc., 2021.

Joel Ye and Chethan Pandarinath. Representation learning for neural population activity with Neural Data Transformers. Neurons, Behavior, Data analysis, and Theory , August 2021. ISSN 2690-2664. doi: 10.51628/001c.27358.

Joel Ye, Jennifer Collinger, Leila Wehbe, and Robert Gaunt. Neural Data Transformer 2: Multi-context Pretraining for Neural Spiking Activity. In Advances in Neural Information Processing Systems , volume 36, pages 80352-80374. Curran Associates, Inc., 2023.

Hamidreza Abbaspourazad, Eray Erturk, Bijan Pesaran, and Maryam M. Shanechi. Dynamical flexible inference of nonlinear latent factors and structures in neural population activity. Nature Biomedical Engineering , 8(1):85-108, January 2024. ISSN 2157-846X. doi: 10.1038/s41551-023-01106-1. Publisher: Nature Publishing Group.

Omid G. Sani, Bijan Pesaran, and Maryam M. Shanechi. Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks. Nature Neuroscience , 27(10): 2033-2045, October 2024. ISSN 1546-1726. doi: 10.1038/s41593-024-01731-2. Publisher: Nature Publishing Group.

Steven Diamond and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research , 17(83):1-5, 2016.

Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision , 5(1):42-60, 2018.

Mark M. Churchland, John P. Cunningham, Matthew T. Kaufman, Justin D. Foster, Paul Nuyujukian, Stephen I. Ryu, Krishna V. Shenoy, and Krishna V. Shenoy. Neural population dynamics during reaching. Nature , 487:51-56, 2012. ISSN 00280836. doi: 10.1038/nature11129.

Christian Y Song, Han-Lin Hsieh, Bijan Pesaran, and Maryam M Shanechi. Modeling and inference methods for switching regime-dependent dynamical systems with multiscale neural observations. 2022. doi: 10.1088/1741-2552/ac9b94.

Daniel W. Moran and Andrew B. Schwartz. Motor cortical representation of speed and direction during reaching. Journal of Neurophysiology , 82(5):2676-2692, 1999. doi: 10.1152/jn.1999.82.5.2676. PMID: 10561437.

Shy Shoham, Liam M Paninski, Matthew R Fellows, Nicholas G Hatsopoulos, John P Donoghue, and Richard A Normann. Statistical encoding model for a primary motor cortical brain-machine interface. IEEE Transactions on Biomedical Engineering , 52, 2005. doi: 10.1109/TBME.2005.847542.

Patrick N. Lawlor, Matthew G. Perich, Lee E. Miller, and Konrad P. Kording. Linear-nonlinear-time-warppoisson models of neural activity. Journal of Computational Neuroscience , 45(3):173-191, December 2018. ISSN 1573-6873. doi: 10.1007/s10827-018-0696-6.

Shinsuke Koyama, Uri T. Eden, Emery N. Brown, and Robert E. Kass. Bayesian decoding of neural spike trains. Annals of the Institute of Statistical Mathematics , 62(1):37-59, February 2010. ISSN 1572-9052. doi: 10.1007/s10463-009-0249-x.

Uri T. Eden, Loren M. Frank, Riccardo Barbieri, Victor Solo, and Emery N. Brown. Dynamic analysis of neural encoding by point process adaptive filtering. Neural Computation , 16(5):971-998, May 2004. ISSN 0899-7667. doi: 10.1162/089976604773135069.

Skipper Seabold and Josef Perktold. statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference , 2010.

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in python. Journal of Machine Learning Research , 12:2825-2830, 2011.

João D. Semedo, Anna I. Jasper, Amin Zandvakili, Aravind Krishna, Amir Aschner, Christian K. Machens, Adam Kohn, and Byron M. Yu. Feedforward and feedback interactions between visual cortical areas use different population activity patterns. Nature Communications , 13(1):1099, March 2022. ISSN 2041-1723. doi: 10.1038/s41467-022-28552-w.

A Appendix

A.1 Derivation

Here we provide the derivation for PGLDM (prioritized generalized-linear dynamical modeling), a covariance-based subspace identification algorithm that learns a dynamical model of a primary time-series while dissociating shared vs. private latents with a secondary time-series. We define the following equivalent formulation for our dynamical model (equation (6)), where the block structure delineates shared ( x (1) ) and private ( x (2) , x (3) ) latent states

with parameters and noise terms defined as in sections 2.1-2.2 and 3.1.

A.1.1 Standard Covariance-Based SSID

Before we present the derivation for PGLDM, we review a few steps in standard covariance-based SSID (section 2.1) that will help us in the derivation. First, it can be shown that the τ -th lag crosscovariance terms for r can be written in terms of model parameters as Λ r τ = Cov( r k + τ , r k ) = C r A τ -1 G , where G := Cov( x k +1 , r k ) . Using this relationship, the Hankel matrix, H r , can be expanded as [19, 20]

Second, using a singular-value decomposition the above Hankel matrix H r can be decomposed into observability, Γ r , and controllability, ∆ , matrices from which model parameters can be extracted [19, 20]

We note that there exists a canonical correlation analysis (CCA) version of this algorithm wherein H r is left and right normalized by the square root matrices of the future and past observation covariance

matrices, respectively, prior to the singular-value decomposition. Specifically L -1 H r M -T SVD = ˆ U ˆ Σ ˆ V T , where Λ ff = Cov( r f , r f ) = LL T and Λ pp = Cov( r p , r p ) = MM T . The observability and controllability matrices can be recovered as Γ r = L ˆ U ˆ Σ 1 / 2 and ∆ = ˆ Σ 1 / 2 ˆ V T M T , respectively.

We refer readers to section 8.7 of Katayama [20] for more detail. We can similarly derive a CCA version of PGLDM, which we present next. This can be achieved by applying the appropriate normalizations to both H r and H zr , the cross-term Hankel matrix defined below, prior to the singular-value decomposition. The resulting decomposition would then have to be un-normalized to retrieve the appropriate observability and controllability matrices.

A.1.2 PGLDM: Stage 1 derivation

In the first stage of our algorithm, our goal is to learn the model parameters that correspond to the shared dynamical subspace of z and r via the latent state x (1) k : ( A 11 , C (1) r , C (1) z , b , d ) . Before deriving the first stage, we first present a few parameter definitions and simplifying assumptions. As noted in section 3.1, G can be partitioned as G = [ G (1) T G (2) T G (3) T ] T due to the block structure of equation (13). Further, based on assumptions 3.2 and 3.3 we can set G (3) = 0 . Leveraging the block structure of G we can also define a simplified block-partitioned structure for the controllability matrix ∆ (equation (15)) as

The block-partition structure of ∆ will play an important role in the separation of learning between stages 1 and 2 and will enable the prioritized identification of shared dynamics A 11 .

To begin the derivation, it can be shown that the τ -th lag cross-covariance between z and r can be written in terms of model parameters as Λ zr τ = Cov( z k + τ , r k ) = C z A τ -1 G . Using the block definition of G , we can simplify the cross-covariance term as

The Hankel matrix between future secondary observations and past primary observations can then be expanded as

A singular-value decomposition of H zr yields the observability matrix for z (i.e., Γ z ) and the controllability matrix ∆ (1) associated with the shared dynamics

At this point, C (1) z can be read off the first n z rows of Γ z . The shared latent dynamics matrix A 11 can be learned by solving a least-squares problem based on the controllability matrix ∆ (1) (as introduced in section 3.2.1)

which has the following closed-form solution: A 11 = ∆ (1) ( ∆ (1) ) † .

To extract C (1) r , we first note that the Hankel expansion in equation (15) can be rewritten with the block-structure of ∆ in mind as (we omit partitions corresponding to latents x (3) due to the decoupling)

where equivalence (a) is due to the block-partition structure of ∆ and equivalence (b) implicitly introduces a block structure to observability matrix Γ r , where Γ (1) r and Γ (2) r correspond to the observability matrices associated with the shared and private latents, respectively. More formally,

where Γ r denotes Γ r from which the top n r rows have been removed. Taken together, H r can be viewed as the sum of 'shared' and 'private' components (equation (19)). Thus, we can compute Γ (1) r as

where we have used the orthonormal property of right singular vectors V to conclude ∆ (2) ∆ (1) † = 0 . At this point, we can extract C (1) r by reading the top n r rows of Γ (1) r . Finally, b and d can both be learned directly from the data either by computing the empirical mean (when working with continuous Gaussian observations) or during the moment transformation (section 2.2). This concludes the learning of all parameters associated with the shared dynamical subspace, i.e., ( A 11 , C (1) r , C (1) z , b , d ) .

A.1.3 PGLDM: Stage 2 derivation

In the second stage of our algorithm, our goal is to learn model parameters that describe the private dynamics of r via the latent state x (2) k : ([ A 21 A 22 ] , C (2) r ) . To learn these parameters, we first extract the private component in equation (19), termed H (2) r , by subtracting Γ (1) r ∆ (1) from H r ,

and decompose it via a singular-value decomposition to get Γ (2) r and ∆ (2) as (we omit reference to private latents x (3) due to the decoupling)

At this point, we take C (2) r as the top n r rows of Γ (2) r and concatentate with C (1) r to complete C r . To complete the state dynamics matrix A , we refer back to the block-structure representation of the controllability matrix in equation (16)

from which we construct the following relationship

where ∆ and ∆ are defined as in equation (18). We can further isolate the residual state transitions as the solution to the following equation (taken from the second row of equation (23))

which has the following closed-form least-squares solution: [ A 21 A 22 ] = ∆ (2) ∆ † . The full state dynamics is the concatenation A = [ A 11 0 A 21 A 22 ] . This concludes the learning of parameters for the private dynamics in r , i.e., ([ A 21 A 22 ] , C (2) r ) .

A.1.4 PGLDM: Stage 3 derivation

Finally, in the third state we can learn the model parameters associated with the private dynamics in z , that is ( A 33 , C (3) z ) , in an approach similar to stage 2. We first construct a future-past Hankel matrix, H z , associated with the secondary observation z . Using an analysis similar to equations (19) and (21), we can show that H z as the sum of 'shared" and 'private" components. In the case of secondary observation z , we can explicitly show that its observability matrix, Γ z , assumes a block form as

where equivalence (a) is due to the block-partition definitions of C z and A in equation (13). Henceforth, we omit the middle block of Γ z , without any loss of generality, and simplify H z as

where equivalence (a) used the block structure of Γ z and equivalence (b) implicitly introduced a block structure on the controllability matrix associated with observation z . In stage 1 we had learned

Γ (1) z , thus we can use an approach similar to equations (21)-(22) to extract the private component of H z . We first compute ∆ (1) z , the z controllabilty matrix associated with the shared dynamics, as

Then we can extract and decompose the private component of the Hankel matrix

As in the previous stages, C (3) z is taken as the top n z rows of Γ (3) z , thus completing C z . To learn A 33 we use the same approach as standard covariance-based SSID (section 2.1) and solve the problem Γ (3) z = Γ (3) z A 33 , where Γ z and Γ z denote Γ z from which the top or bottom n z rows have been removed, respectively. This optimization problem has the closed-form least-squares solution A 33 = Γ † z Γ z . This concludes the learning of parameters for the private dynamics in z , i.e., ( A 33 , C (3) z ) .

A.1.5 Transformation of joint Gaussian and Poisson moments

In sections A.1.1-A.1.4 we demonstrated how all model parameters can be extracted in three stages with prioritization, starting from the second-moments of the r and z . As an example of how these moments can be computed from generalized-linear observations, we consider the specific case of Poisson/Gaussian observations as y and z , respectively. The joint second-moment Λ z fm r pn can be analytically recovered from the computable moments of y (the Poisson observations) and z (the Gaussian observations), using equation (11) as

where m and n indicate index-wise notation. Here we provide a sketch of the proof. Without loss of generality, assume z and r are stationary with a mean of 0 (e.g., demeaned during preprocessing). We can compute the covariance of any two elements j and k of vectors z f and y p respectively as

where (a) is because z f j and y p k are independent when conditioned on latent log-rate r p k . Next, we use the fact that z f and r p are jointly Gaussian random processes and, as a result, the mean of the conditional distribution, E [ z f j | r p k ] , is equal to Λ z f j r p k Λ -1 r p kk r p k (i.e., the linear least-square estimate of z f j using r p k ). The last step is to compute the expectation

which, after rearranging terms, yields

We note that the final equation is equivalent to a derivation provided by Buesing et al. [21] as their supplementary equation (6) to compute cross-covariances between Poisson observations and Gaussian inputs , instead of between joint Poisson and Gaussian observations (as was in our case). The remaining unimodal (i.e., Poisson-only) moment conversions that are required to compute H r are performed per equation (5) in section 2.2. For Bernoulli-Gaussian moment conversion equations, we refer the reader to section 3.3 equations (5) and (6) in Stone et al. [24].

A.1.6 Generalized cross-term Hankel matrix with different horizons per observation

For ease of exposition, the derivation in section A.1.2 was provided for a cross-term Hankel matrix H zr that was formed with equal horizons for z and r as

In general, the rank of Hankel matrices formed from ideal data covariances can be shown to be the same as the state dimension associated with it [19, 20], i.e., n 1 = rank ( H zr ) per equation (17) and n 1 + n 2 = rank ( H r ) per equation (15). However, during system identification these Hankel matrices are formed from non-ideal empirical sample covariances and, as a result, are typically full rank. Nevertheless, we expect the singular values associated with real dynamics (e.g., the first n 1 singular values in H zr ) to be larger than subsequent singular values that are due to noise. Indeed, the goal of the SVD applied to Hankel matrices, e.g., in equations (15), (17), and (22), is to remove noisy singular values and only keep the largest singular values that are most likely due to real dynamics.

Given that the Hankel matrices formed during system identification are typically full rank, their rank is determined based on their dimensions, i.e., rank ( H zr ) = min ( i × n r , i × n z ) and rank ( H r ) = i × n r . Thus, the horizon parameter i that is used to form the Hankel matrix plays an important role in its final dimensions, rank, and, consequently, on the maximum number of non-zero singular values that can be preserved after applying SVD. This, in turn, determines the maximum state dimension that can be learned for the resulting model. Thus, to provide more flexibility over the state dimensions that can be learned in each stage of PGLDM, we generalize the Hankel matrix H zr to support different horizon values for each of the observations, i z and i r , such that

The observation horizon i r is also used when forming the Hankel matrix H r , per equation (14). The additional flexibility gained from having different horizon values can be especially critical in scenarios wherein the dimensionalities of z and r are very different, such as in the case of our NHP analysis in section 4.2, where n z = 4 and n r = 15 . We select the final horizons i z and i r via an inner cross-validation based on which values achieve the best decoding accuracy in the training data.

A.2 Assumptions and generalizability of the block structure formulation

̸

Here we explain the assumptions underlying the proposed block formulation of A , as defined in equation (7), and discuss the model's generalizability. First, we assume that the private latents of y (i.e., x (2) ) and of t (i.e., x (3) ) do not drive the shared latents (i.e., x (1) ) so as not to leak private dynamics into shared dynamics. Thus, we take A 12 = 0 and A 13 = 0 . Second, we assume that the private dynamical latents of the two signals (i.e., x (2) and x (3) ) do not drive each other, and so we take A 23 = 0 and A 32 = 0 . Finally, the asymmetrical decision to set A 21 = 0 and A 31 = 0 was dictated by the asymmetrical roles played by the primary (i.e., y ) and secondary (i.e., t ) time-series in our formulation. We initially made the assumption that A 31 = 0 to simplify the derivation of our method because it allowed the optional third stage (i.e., extraction of the dynamics private to t ) to operate independently of the first stage (i.e., extraction of the shared dynamics). However, this assumption was still in line with our notion of shared and private dynamics; if A 31 = 0 , then the primary time-series ( y ) would be informative (i.e., predictive) of the private dynamics in the secondary time-series ( t ) through the influence of the shared dynamics x (1) on future x (3) .

̸

̸

Although A 21 fundamentally has a similar information leakage impact in the reverse direction, we chose to keep the more general form of A 21 = 0 . Unlike the A 31 case, a non-zero A 21 does not couple the multi-stage learning. This is because we designate a primary time-series in our formulation (taken as y here) that acts as a predictor for the secondary time-series (taken as t here). Given this designation, we exclusively calculate all Hankel matrices using the primary/predictor time-series as the past observations. Specifically, we only compute Λ zr τ = Cov( z k + τ , r k ) (equation (8)) and not Λ rz τ = Cov( r k + τ , z k ) . From a derivation perspective, this means that A 21 only appears in stage 2 and not in stage 1, thereby never coupling the learning of private and shared dynamics. However, we clarify that if the user requires a strictly symmetrical definition of A wherein all latent states are fully decoupled from each other (i.e., a block-diagonal structure), our algorithm can, with a minimal modification to stage 2, cover the case where A 21 = 0 . To learn a block diagonal A , one would only need to compute the solution to ∆ (2) = A 22 ∆ (2) instead of the original formulation ∆ (2) = [ A 21 A 22 ] ∆ (1 , 2) (see section 3.2.2 and equation (24)).

̸

Finally, taking A 21 = 0 also has particular significance in the case when there are no private latents associated with t , that is when n 3 = 0 . In this case, taking A 21 non-zero allows the block-structured state-space form defined in equations (6) and (7) to be assumed without any loss of generality. Specifically, when n 3 = 0 , the latent states in our model describe the primary time-series y with a subset also explaining the secondary time-series t . Formally, we define the true dimensionality of the shared states (denoted by n 1 ) based on the rank of the observability matrix for the pair ( A , C z ) . It can be shown using linear systems theory that an invertible linear transformation of the latent states (i.e., a similarity transformation) always exists that can place the n 1 dimensional latent subspace that is observable via t as the first few dimensions of the latent space, thus giving the block-structured formulation of equation (7). This can be seen by applying Theorem 3.8 from Subspace Methods for System Identification [20] to the first two lines of equation (6). As a result, the blocked formulation of equation (7) is equivalent to the formulation from (6) and we can aim to learn our model in the form of equation (7) without any loss of generality. Thus, our choice to keep A 21 = 0 was also in part motivated by a desire to maintain a more general state-space form for the case when n 3 = 0 .

A.3 Noise statistics

Standard SSID algorithms (e.g., section 2.1) learn linear state-space models of the following form

where state noise, w k , and observation noise, v k , are typically additive Gaussian noise and may have a non-zero instantaneous cross-covariance S = Cov( w k , v k ) . SSID in general does not assume any restrictions on the noise statistics. However, the Poisson observation model (equations (4) and (6)) has no additive Gaussian noise for r k and instead exhibits Poisson noise in y k when conditioned on r k . This means that v k = 0 in equation (6), and thus R = 0 and S = 0 . Imposing these constraints is important for accurate parameter identification for Poisson observations, but was not previously addressed by Buesing et al. [21]. Thus, we require our algorithm to find a complete parameter set Θ ′ that is close to the learned ( A , C r , C z , b ) by PGLDM and imposes the noise statistic constraints R = 0 and S = 0 . To do this, inspired by Ahmadipour et al. [22], we form and solve the following convex optimization problem to satisfy the noise statistics requirements

where Λ x := Cov( x k , x k ) denotes the latent state covariance. Further, we enforce the following covariance relationships, derived from equation (1) [19], as constraints

This approach has multiple benefits. First, it finds noise statistics that are consistent with the assumptions of the model (e.g., R = 0 ). Second, it enforces the validity of learned parameters, i.e., parameters corresponding to a valid positive semidefinite covariance sequence (see section 4.4). It also enables state prediction (see appendix A.8.2). Combining the previously found parameters and the matrix Q that corresponds to the minimizing solution Λ x of equation (30), we have the full parameter set Θ ′ = ( A , C r , C z , b , Q ) . We used Python's CVXPY package to solve the semidefinite programming problem defined in equation (30) [39, 40]. For all of our comparisons against PLDSID, we learned the noise statistics associated with the method's identified parameters using this approach, but keeping the rest of the algorithm the same.

A.4 GLDMparameter equivalencies

Here we briefly discuss the equivalence of learnable parameters by SSID for linear dynamical models of the form in equation (1). In section 2.1 we stated that standard covariance-based SSID learned the parameters Θ = ( A , C r , G , Λ r 0 ) , but not any of the noise statistics (e.g., covariances Q and R ). Indeed, there also exist SSID approaches that instead learn the following parameter set ( A , C r , Q , R , S ) [19, 20]. Both sets of parameters are valid and knowledge of either completely

̸

defines the model in equation (1). Moreover, the two sets of model parameters can be related to each other using equations (31) [19] - the same relationships we use as constraints in the convex optimization problem defined in appendix A.3 for learning the noise statistics of the PLDS model (4).

A.5 Possibility of learning unstable modes in small data regimes

Subspace identification methods generally only converge to the correct system parameters asymptotically (see figure 4 in appendix A.11), as the empirically estimated covariances also converge to their true values [19]. For finite samples, however, there will always be some error in the learned parameters. Although such errors are generally benign, extreme scenarios can result in unstable state dynamics, i.e., the identified A has at least one eigenvalue with magnitude larger than 1. We omitted any learned unstable models in our analyses, which was reflected in the reduced number of samples in the standard error of the mean (s.e.m). However, we rarely encountered unstable models; for example, in the results presented in figure 4, there were no unstable models for training set sizes typical of neuroscience datasets (i.e., 1e5 or 1e6 training samples).

A.6 Possibility of accumulation of error in multi-staged learning

There are two aspects of the learning to consider. First, there is a signal-to-noise (SNR) consideration that is fundamental to learning methods in general. In stage 1, our method requires high SNR in the cross-correlations between future secondary time-series observations and past primary time-series observations. If most of the signal present in the primary time-series is not attributable to the shared latent states (i.e., the residual Hankel in equation (10) dominates), then most of H zr 's singular values will be small and stage 1 may have greater estimation error. If the reverse situation holds and the shared latent states explain most of the signal in the primary time-series, then the residual Hankel, H (2) r will mostly have small singular values, possibly resulting in estimation errors during stage 2. Inspection of the singular values prior to model parameter extraction can, however, help guide method usage. For example, if the singular values of H zr are small, then this may indicate that the two time-series do not have shared dynamics and only stage 2 of the method is needed (i.e., standard GLDM). If the singular values of the residual Hankel are small, then this may indicate that almost all dynamics are shared and only stage 1 is needed.

Asecond consideration is a numerical one that could result in an accumulation of errors in downstream stages. If during stage 1 the modes are identified inaccurately because there is minimal shared dynamics between the two time-series and/or the training sample size is too small, then this estimation error could impact the computation of the residual matrix H (2) r thereby introducing error in stage 2. Examining the singular values of H zr may, however, also help avoid this situation. Although the multi-stage learning can lead to error accumulation in some situations, it comes with the benefit that our method has the ability to more accurately identify the shared dynamics (when they exist) in stage 1, compared to existing GLDM methods. Moreover, Figures 1b, 2b, and 3b suggest that the impact of error accumulation is not severe and can be situation-dependent; for example, self-prediction performance of PGLDM reaches that of PLDSID in 1b, reaches very close to it in 2b, and exceeds it in 3b.

A.7 Experimental details

Below we provide details on how simulations were generated and about our real data analyses.

A.7.1 Simulations

For our synthetic data in section 4.1, we simulated generalized-linear observations from random models as per equation (6). For the simulations used to generate the results in Table 1, we fixed the number of shared and private latent states as n 1 = 2 , n 2 = 6 , and n 3 = 4 . We randomly selected the observation dimensions with uniform probability from the following ranges: 10 ≤ n r ≤ 15 and either 5 ≤ n z ≤ 10 , when the secondary observation was Gaussian, or 10 ≤ n z ≤ 15 , when Poisson. For the simulations used to generate figure 2, we fixed the latent dimensions as n 1 = 4 , n 2 = 12 , and n 3 = 4 and randomly sampled observation dimensions with uniform probability from: 20 ≤ n r ≤ 30 and 5 ≤ n z ≤ 10 . Across all simulations we used these dimensions to generate random model parameters Θ = ( A , C r , C z , b , d , Q ) . We constrained the complex eigenvalues (i.e.,

modes) of the state transition matrix A to have magnitudes uniformly distributed between [0 . 93 , 0 . 99] and phases uniformly distributed between [0 . 019 , 0 . 314] . These restrictions correspond to stable, slow-decaying systems with time-constants within [0 . 138 , 0 . 995] seconds and frequencies within [0 . 3 , 5] Hz that are representative of various real time-series data, such as neural dynamics [41, 42].

For all Gaussian simulations we set the time-series mean (e.g., b or d ) to 0. Additive Gaussian noise covariances (e.g., Q ) were randomly generated to be positive definite matrices. For observation timeseries, the observation matrix C was generated to hit a target signal-to-noise, defined as the variance associated with latent states normalized by observation noise variance (e.g., ( C z Λ x C T z ) / ( Λ ϵ ) ). Target SNR values were fixed at 10 for the simulations used in Table 1 or were randomly generated as 10 α with α uniformly distributed between [0 , 2] for the simulations in figure 1. All Poisson simulations were generated on a 10 ms timescale with a baseline log rate (e.g., b ) randomly selected within [0 . 5 , 15] Hz. Observation matrix C was scaled to achieve a desired per-dimension 95th percentile modulation depth of 10, where modulation depth is defined as exp( C x k ) . All Bernoulli observations were also generated on a 10ms timescale with a baseline set to 0. In this scenario, the observation matrix C was scaled such that the corresponding Gaussian latent defined by C x k would have a standard deviation approximately equal to 1.

To simulate a state noise covariance Q that adhered to the block-structure assumption defined in assumption 3.2, we leveraged the fact that latent x (3) is completely decoupled from latents x (1) and x (2) in our model definition (as per equations (7) and (13), and assumptions 3.2 and 3.3). We chose to simulate the dynamics private to the secondary observation time-series as a separate 4-dimensional latent dynamical model. Specifically, we chose to generate one dynamical model corresponding to the latents x (1) and x (2) , and a separate dynamical model corresponding to x (3) . This approach, however, only works for Gaussian secondary observations. For the Poisson-Poisson case we instead generated an unconstrained Q . Despite this deviation from assumption 3.2, our method's ability to identify the shared dynamical modes was not impacted, as shown by the results in Table 1.

Finally, for all simulation experiments we use horizon values of 10 for both observations by default. For the simulations used in Table 1 we generated 25600 training samples, and for the simulations used in figure 1 we generated 2e6 samples, splitting 50/50 into training and test data.

A.7.2 NHP Dataset 1: motor cortex recordings during reaching

All NHP analyses in figure 2 were performed on a public dataset released by the Sabes lab [17], using the following sessions from monkey I: 20160915/01, 20160916/01, 20160921/01, 20160927/04, 20160927/06, 20160930/02. We performed cross-validation using randomly-selected, non-overlapping subsets of 15 channels ( n r = 15 ) binned at 50ms resolution within each session. We used a nested inner cross-validation to select hyperparameters per fold based on the prediction CC of kinematics in the training data. Hyperparameters in this context were discrete horizon i r , continuous horizon i z , and time lag, which specifies how much the neural time-series should be lagged to time-align with the corresponding behavioral time-series [33, 43, 44]. We swept i r values of 5 and 10 time bins, i z values of 10, 20, 22, 25, 28, and 30 time bins; and lag values of 0, 2, 5, 8, and 10 time bins. We removed channels that had average firing rates less than 0.5 Hz or greater than 100 Hz. Similar to Lawlor et al. [45], we also removed channels that were correlated with other channels using a correlation coefficient threshold of 0.4.

A.7.3 NHP Dataset 2: visual areas V1-V2 during stimulation

All NHP analyses in figure 3 were performed on a public dataset released by Zandvakili and Kohn [16, 18], using a randomly selected session 107l002p67. In the released dataset, each recording session consisted of repetitions of 1.28s of stimulus presentation (1 of 8 possible orientation gratings) followed by 1.5s of a blank screen. For our analysis, we only considered neural activity during periods of stimulation binned at 20ms resolution. Further, we performed five-fold cross-validated modeling within each stimulation condition. For both observation time-series, we removed channels that had average firing rates less than 0.5 Hz or greater than 100 Hz. Because there were more V1 units than V2 units, we performed cross-validation using randomly-selected, non-overlapping subsets of V1 units equal to the number of available V2 units, that is n r = n z = | V2 units | . We present averaged results across folds, stimulation conditions, and non-overlapping subsets, which we collectively refer to as 'trials" for a total of 120 trials. No hyperparameter optimization was performed for this analysis; discrete horizons for SSID-based methods were set to a fixed value of i r = i z = 5 .

A.8 Model evaluation

We evaluated learning using two different metrics: (1) shared dynamical mode identification for models with the true shared latent state dimensionality and (2) predictive power of inferred latent states.

A.8.1 Shared dynamical mode identification accuracy

To evaluate the learning of shared dynamics, we computed the normalized eigenvalue error as ∥ Ψ true -Ψ id ∥ 2 / ∥ Ψ true ∥ 2 , where Ψ true and Ψ id denote vectors containing the true (i.e., the eigenvalues of A 11 ) and learned shared eigenvalues, respectively, and ∥ · ∥ 2 denotes the Euclidean L2-norm. To compute the first metric for our baselines that did not explicitly model shared dynamics, we needed to select the n 1 modes identified from the primary time-series only that were the most representative of the secondary time-series. To do so, we first trained our baselines on the primary time-series observations and extracted the latent states. Then, we sorted these learned latent states based on their accuracy in predicting the secondary observations (appendix A.8.2). We computed the eigenvalues associated with the top n 1 most predictive latent states, which we considered as the shared modes identified by our baselines. This was only necessary for configurations where the learned latent dimensionality was greater than the shared dimensionality (i.e., n 1 + n 2 > n 1 ). For configurations wherein learned n x is smaller than true n 1 , we substituted missing modes with 0 prior to computing the normalized error.

A.8.2 Predictive power of inferred latent states

We also evaluate the predictive power of our learned models in two scenarios: (1) prediction of the secondary observation time-series from the primary time-series, a common use-case in neuroscience [2, 7, 46], and (2) one-step ahead self-prediction of the primary time-series from itself. Using models learned from the training data, we constructed recursive Bayesian filters to estimate the latent states in a test dataset. Because our analyses in figures 1-3 used Poisson observations as the primary time-series, we chose to use a Poisson point-process filter (PPF) [47] for state estimation. (We note that using the PPF for state estimation is only possible if the learned noise statistics are valid, see appendix A.3.) We denote the one-step ahead latent state prediction of x k using all samples of y k up to time k -1 by ˆ x k | k -1 . These state estimates can be used to predict both sets of (latent) observations as either C z ˆ x k | k -1 or C r ˆ x k | k -1 .

In order to compare the predictive power of our learned models with those of our baselines (PLDSID and Laplace-EM), it was necessary to learn an observation model (e.g., C z ) for the secondary timeseries. To do so, we first estimate the latent states in the training data using a PPF and then fit a regression model (scikit-learn; statsmodels) from the latent states to the secondary observation [48, 49]. For example, when the secondary observation, z k , is a continuous Gaussian time-series, the parameter C z was learned using ordinary least-squares such that C z = Z ˆ X T ( ˆ X ˆ X T ) † , where columns of Z and ˆ X contain z k and ˆ x k | k -1 for all training timepoints k . To make all methods more comparable, we use the same approach to refit the secondary observation's model learned by PGLDM.

For continuous Gaussian observations, we quantify the decoding performance using correlation coefficient (CC). For discrete Poisson observations we instead evaluate prediction using the area under the receiver operating characteristic curve (AUC). Since all Poisson predictions were made using the recursive Bayesian filter's estimates of the latent states, our goal was to validate if our model could accurately predict the occurrence of point process events (versus no events) in a given time step when using all past observations of the primary time-series y k . For example, in the results shown in figure 2b-c we computed the probability of an event for the m -th dimension of y at time k conditioned on all observations y 1: k -1 , as

where in (a) we simplify using y k 's conditional independence from the past y 1: k -1 , in (b) we simplify based on y k | x k ∼ Poisson ( exp ( r k )) , in (c) we use the Taylor series approximation of exp ( exp ( r k )) for small exp ( r k ) , and (d) is simply the mean of a log-normal random variable. Note that ˆ r k = C r ˆ x k | k -1 + b and Λ ˆ r = C r Λ ˆ x k | k -1 C T r , where ˆ x k | k -1 is the current estimate for the state and Λ ˆ x k | k -1 the estimate for the state-prediction covariance. We can similarly compute AUC for the prediction of secondary Poisson observations, as in figure 3.

A.9 Stage 3 simulation results

Wevalidated our algorithm's third stage for learning the dynamics private to the secondary observation z using simulations. We simulated 10 random models with the following dimensions n 1 = n 2 = n 3 = 4 , n r = 20 , n z = 4 , with the primary/secondary observation pair being Poisson/Gaussian. The Gaussian observations were set to have SNR of 10 (see appendix A.7.1). We constrained the complex eigenvalues of the state transition matrix A to have magnitudes uniformly distributed between [0 . 93 , 0 . 99] (as before) and phases uniformly distributed between [0 . 019 , π ] . Using a training set size of 1e6 samples, we used PGLDM with all three stages to learn all sets of shared and private modes. We then computed the normalized eignevalue error between the ground truth private modes (i.e., eigenvalues of A 33 ) and the identified private modes. We found an average normalized identification error of 1.12% for learning the dynamics associated with the latent states x (3) . For comparison, this is approximately equivalent to -2.05 on the y-axis of figure 1 .

A.10 Computation time details

We compared the computational runtime efficiency between our method PGLDM (using either stage 1 only or both stages 1 and 2), PLDSID, and Laplace-EM on one session of NHP data binned at 50ms resolution (section 4.2 and appendix A.7.2). We repeatedly trained on 25 distinct time-series datasets and computed an average runtime for learning across all fits. Each dataset consisted of a 6097-by-15 matrix (timesteps-by-features) of Poisson observations and a 6097-by-4 matrix of Gaussian observations. All methods learned latent models of dimension 8 (i.e., n x = 8 ). When both stages of PGLDM were used, we fixed n 1 = 4 while keeping n x = 8 . Laplace-EM was run for 100 iterations (the default setting) and learning times are reported both as an average for 100 iterations as well as for a single iteration. For PGLDM and PLDSID we separately report average running times for the SSID portion of the algorithm and for the convex optimization problem used to learn state noise, as outlined in appendix A.3. The results are presented in Table 2.

Most of the computational cost of our algorithm is involved in the matrix operations associated with 1) computing the necessary covariance/Hankel matrices, 2) performing the moment conversion, and 3) performing the SVD of the future-past Hankel matrices. To perform the moment conversion our method requires a covariance matrix for stacked future-past Poisson-Poisson observations (section 2.2) and a future-past Gaussian-Poisson Hankel matrix (section 3.2.1). Both of these empirical estimates of second-order covariances are computed using matrix multiplications which scale with the number of samples. As an example, we can consider the setup used for the computational cost analysis in Table 2, wherein n r = 15 and i r = 10 (horizon). The computed square Poisson-Poisson covariance matrix was of dimension 2 ∗ n r ∗ i r = 2 ∗ 15 ∗ 10 and was the result of a matrix multiplication between two matrices of dimension (2*10*15)-by-6078, where 6078 = timesteps -2 ∗ i r +1 . Thus, this operation would scale linearly with the length of the training data. Similarly, the computational cost of this matrix multiplication scales linearly with feature dimension and horizon. The remaining operations (i.e., the SVD and the moment conversion itself) are functions of the latent-state dimension and the feature dimensions for each observation timeseries.

All running time analyses were performed on a 2020 Macbook Pro using 2 GHz Quad-Core Intel Core i5 CPU with 16GB of 3733 MHz RAM.

A.11 Data Efficiency

We also investigated the data efficiency of our method by generating 20 random Poisson/Gaussian systems and studying the effect of training set size on learning. The random systems used here were generated using the same procedure as the systems used in figure 1 except for the latent dimensionalities of the primary signal. We instead uniformly sampled the total latent dimensionality of r as 1 ≤ n 1 + n 2 ≤ 10 and the shared dimensionality as 1 ≤ n 1 ≤ n 1 + n 2 ; we kept a fixed size for the secondary observations private states n 3 = 4 . We used 1e3, 1e4, 1e5, or 1e6 samples to train models, and then tested them on 1e6 samples of independent held-out data (figure 4). We found that with increasing training set sizes the performance of our method improved, both in identification of shared dynamics and in overall predictive power. This is expected because empirically estimated covariances converge towards their true values with increasing training set sizes, thereby improving the overall performance of our algorithm.

(a)

(q)

0.80

(c)

Prediction Correlation

0.80

AUC

NormalizedSharedMode

(01601)

-2.2

Coefficient(CC)

Self-prediction

0.78

IdentificationError

0.76

-2.6

Oraclemodel

Oraclemodel

0.76

performance

-3.0

0.72

performance

-3.4

0.74-

3.0

4.0

5.0

6.0

3.0

4.0

5.0

6.0

3.0

4.0

5.0

6.0

Training sample size(log10)

Training sample size(log10)

Training sample size(log10)

A.12 V1-V2 Results (time-series designation swap)

For completion we also present the complementary analysis to the results in figure 3, wherein we take neural population spiking activity from V1 as y k and activity from V2 as t k . The rest of the analysis is kept the same as described in sections 4.3 and A.7.3, and the results are presented in figure 5. In this scenario, PGLDM does not make a substantial difference in predicting V2 activity from V1 activity when compared with PLDSID and Laplace-EM. Prior work using this same dataset has shown that the feedback direction, that is the V2 (past) to V1 (future) direction, generally exhibited higher correlations between the two time-series as compared to the feedforward direction [50]. We hypothesize that this finding might also explain the differences that we see in our results (i.e., the differences between figures 3 and 5).

(a)

(b)

0.540-

AUC

0.70

AUC

0.535

self-prediction

89'0

decoding

0.530

0.66

PGLDM

0.525

PLDSID

0.64

0.520

n1=6

Poisson

Laplace-EM

0.62

n1=6

F09'0

0.515

2

4

6

8

2

4

6

8

Number of Latent States(nx)

Number of Latent States(nx)

(c)

(d)

F09'0

AUC

0.85.

AUC

F85'0

0.80

self-prediction

0.56

decoding

0.75

0.54

工

0.70

0.52-

0.65

0.50

N

0.48

0.60

0.55

0.46 -

nx=8

nx=8

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We state that our aim in this work is to improve upon existing GLDMs, and their associated learning algorithms, that only consider a single generalized-linear timeseries and therefore cannot dissociate shared vs private dynamics in two generalized-linear time-series. We derive and present a novel algorithm with this capability and demonstrate in both simulations and real data, as stated in the abstract.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have a specific subsection called 'Limitations" wherein we discuss the limitations of our model assumptions and methodology. We also have a supplementary section that discusses the computational efficiency of our method. Finally, in our experimental details we discuss the number of simulated systems and amount (and type) of real data we have validated our method on.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best

judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have a full derivation of our method, with its associated assumptions, in our appendix. We provide a proof-sketch and outline of our method in the main manuscript.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We believe we have provided all the necessary information regarding simulation configurations, open dataset locations, data preprocessing procedures, model hyperparameters, and cross-validation settings needed to reproduce the results. For the computational running time analysis we have also specified the hardware/compute resources used to run the analysis. Beyond reproducing analysis results, we have also outlined our analytical algorithm step-by-step in thorough detail to enable re-implementation by readers, if desired.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided our current code with our submission and will provide a public camera ready version upon acceptance. We also provide links to the open datasets used as well as to the codebase of one of our baselines.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide details on experimental settings in the main text and in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For all figure and table results we reported averages and their associated standard error of measures. For figure 2 results we also performed significance tests (Wilcoxon) and report statistically significant p-values.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Most of our experimental results are agnostic to the compute resources used. For the computational efficiency results reported in the appendix we do provide information on the compute resources used to run the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have adhered to the conference Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Although our primary intention with this work was to develop a novel SSID method from a foundations perspective, methods like ours have previously found utility in brain-computer interfaces and therefore our work may have positive societal impacts by benefitting individuals with disability.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work does not make use of any models or data that could have a high risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We provide citations and/or links to all existing assets used in the manuscript including: scikit-learn, statsmodels, cvxpy, the Python ssm package, bestLDS package on Github, and both publicly-available non-human primate datasets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: All model derivations and algorithm implementations are described in detail in the manuscript. We have also submitted our code along with a README with instructions and a tutorial that will be publicly released upon publication.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work does not use crowdsourced experiments or research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our work did not rely on data collected from human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes a new linear dynamical system model that aims to model interaction between two time series by explicitly modeling their shared and private dynamics. Furthermore, authors incorporate generalized linear models in their observation model of the proposed dynamical system to handle different kinds of time series typically encountered in neuroscience (e.g., count-time series, real-valued time series etc.). Authors then extend standard covariance-based subspace system identification to learn the parameters of the proposed model and compared the performance of their proposed model with existing baselines on two experimental datasets and a simulation study.\n\nSTRENGTHS:\n- The paper is very well-written and the ideas are introduced in a natural progression making them easy to follow.\n- Authors identify a gap in literature where existing methods do not model multiple time-series while explicitly accounting for shared and private dynamics when the observed time series are not real-valued.\n- Authors propose an interesting dynamical system for modeling two different time series containing shared and private latents, whose parameters can be analytically estimated by extending existing covariance-based subspace system identification algorithms. \n- The proofs provided in the paper are generally correct.\n- Authors show improvement in the models' ability to predict one time series properties from the other time series when using their proposed dynamical system model which models the dynamics of the two time series jointly.\n\nWEAKNESSES:\n- The proof of the claim that the any general $A$, $C_r$, and $C_z$ can be decomposed into the block form structure described in equation (7) is incomplete. The proof provided in Appendix A.2 uses a well-known theorem, which states that for a non-fully observable linear dynamical system characterized by the system matrices $(A,C)$, there exists a non-singular linear transformation $T$ such that the new linearly transformed system having the new system matrices $A'=T^{-1}AT$ and $C'=CT$ have the following block structure:\n$ \nTA^{-1}T = \\begin{bmatrix} A_{11} & A_{12} \\\\\\\\ 0 & A_{22} \\end{bmatrix}, CT = \\begin{bmatrix}0& C_2\\end{bmatrix}\n$, where the pair $(A_{22}, C_2)$ are observable. Authors apply this theorem to the **special case** when the dimension of $x^{(3)}$ is zero, to reduce the matrices $A, C_z$ and $C_r$ to the desired block diagonal form, which is correct. But then, authors state that a similar argument works when the dimension of $x^{(3)}$ is non-zero. I am not sure how this proof of the special case of $x^{(3)}$ having zero-dimension can be extended to the general case of $x^{(3)}$ having non-zero dimension. The argument that the general matrices $A=\\begin{bmatrix}A_{11}& A_{12}& A_{13}\\\\\\\\ A_{21}&A_{22}& A_{23}\\\\\\\\ A_{31}&A_{32}&A_{33} \\end{bmatrix}$, $C_z = \\begin{bmatrix} C_z^{(1)}&C_z^{(2)}&C_z^{(3)}\\end{bmatrix}$, and $C_r=\\begin{bmatrix}C_r^{(1)}&C_r^{(2)}&C_r^{(3)} \\end{bmatrix}$ can always be reduced to the block-diagonal shown in equation (7) does not fall out as a trivial consequence of the theorem stated above, and requires an explicit proof. Naively applying the above theorem using the dimension of the observability matrix pair $(A, C_z)$ would show an existence of $T$ such that the new transformed system has the matrix with the following block structure: $A=\\begin{bmatrix}A_{11}& A_{12}& A_{13}\\\\\\\\ A_{21}&A_{22}& A_{23}\\\\\\\\ 0&0&A_{33} \\end{bmatrix}$, $C_z = \\begin{bmatrix} 0&0&C_z^{(3)}\\end{bmatrix}$.  I would ask authors to provide a rigorous mathematical proof showing that there always exist a non-invertible matrix $T$ such that a general $A, C_z$ and $C_r$ can be reduced to the block-diagonal form shown in equation (7). In the event, authors are unable to provide a proof, I would ask authors to modify the manuscript accordingly to reflect that the block diagonal structure is an assumption on the system model and remove the phrases \"without loss of generality\". In this review, I am going to continue on the assumption that the general matrices $A, C_r$ and $C_z$ cannot be reduced to the block-diagonal form (until shown otherwise by the authors through a rigorous mathematical proof) and the block-diagonal form is an assumption on the dynamical systems the authors are trying to estimate.  \n- The claims of the papers that the proposed model can be learnt for any generalized-linear process are overstated. To be able to learn the proposed models' parameters (described in equation (6)), the moment conversion equations described in section 3.2.3 are needed. Authors only provide the moment conversion equations for Gaussian, Poisson, and Bernoulli distributions. It is unclear how the model parameters can be learnt when, for example, $\\mathcal{P}_{y|r}(y_k; g(r_k))$ is an exponential, gamma, or inverse-gaussian distribution. It may be the case that a general moment conversion equation may not exist for any arbitrary exponential distribution. Accordingly, I would ask the authors to town down their claims or provide a general moment conversion equation for any arbitrary exponential family distribution.\n\nMinor\n\n- Please do not use pythonic notation (without defining it first) for mathematical vector operations. For example in equation (11), R.H.S. of the equation is dividing a matrix by a vector. That is not a well-defined operation.  Similarly, equation (12) the operation $\\mu_{t_{f_m}}\\mu_{y_{p_n}}$ does not make sense, it should be $\\mu_{t_{f_m}}\\mu_{y_{p_n}}^T$. Please also state that $\\ln(\\cdot)$ in those operations is applied elementwise, as there exists well defined ways to define matrix functions of $\\ln(\\cdot)$ too.\n\nQUESTIONS:\n- The structure of the proposed $A$ matrix is a bit unintuitive and does not seem to satisfy intuitive properties one would expect from a dynamical system modeling shared and private latents. Intuitively, I would have expected the structure of the $A$-matrix to have the forms of these kinds: $A=\\begin{bmatrix}A_{11}&0&0\\\\\\\\ 0&A_{22}&0 \\\\\\\\ 0&0&A_{33}\\end{bmatrix}$, where the dynamics of the shared latent $x^{(1)}$  and the dynamics of the private latents $x^{2}$ and $x^{3}$ do not affect each other or the structure $A=\\begin{bmatrix}A_{11}&A_{12}&A_{13}\\\\\\\\ A_{12}&A_{22}&0 \\\\\\\\ A_{31}&0&A_{33}\\end{bmatrix}$, where the dynamics of the private latents $x^{3}$ and $x^{2}$ would only affect each other through the shared latent $x^{(1)}$. The structure proposed by the authors, i.e. $A=\\begin{bmatrix}A_{11}&0&0\\\\\\\\ A_{12}&A_{22}&0\\\\\\\\ 0&0&A_{33}\\end{bmatrix}$ has an inherent asymmetry in-built to its dynamics where the dynamics of the second private latent $x^{(2)}$ is affected by the shared latent state but the private latent $x^{(3)}$'s dynamics are only affected by itself. Why is this modeling choice justified? Why dynamics of only one of the private latent can be affected by the shared latent. Is there a domain-specific reason?   \n- Why does PGLDM always perform worse at self-prediction compared to other baselines (Figure 1 (b), 2(b), 3(b))?\n- I am also curious to know that how the results look when decoding the activity V2 from V1, and V1 self-prediction using the system estimated by PGLDM in Section 4.3\n\nLIMITATIONS:\nAuthors have provided a section discussing some limitations. A major limitation of this work that authors do not mention is the comparison with non-linear models which are also being used for decoding neural activity (such as neural data transformer by Ye & Pandarinath'21). I do not think a experimental comparison between these transformer architecture and the linear models proposed in this work is crucial, as the goal of these models have widely diverged with linear models being used more for understanding underlying neural dynamics, whereas the transform models being used for boosting decoding performance. Regardless, I think a mention of these non-linear methods should be made in the limitation sections along with the acknowledgement of a lack of comparison with these methods.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn their study \"Spectral Learning of Shared Dynamics Between Generalized-Linear Processes\", the authors introduce a multi-stage spectral algorithm for learning the parameters of a model for two generalized-linear time-series with shared latent dynamics. Assuming the latent dynamics to contain both shared and private latent dimensions, the authors show that the shared dynamics can be identified from a decomposition of a Hankel matrix formed from the time-lagged covariances between the time series. Additional (second and third stage) analyses can be used to identify the latent spaces private to either of the two time-series. Thanks to moment-conversion schemes developed here and in previous work, their spectral method can be applied to time-series with Gaussian-, Bernoulli- and Poisson-distributed observations. On synthetic data and two datasets with primate neural recordings, the authors show that their method is able to accurately identify the private and shared dynamics. The method appears to perform particularly well on identifying the shared dynamics, leading to better predictions of one time series from the other than tested alternative methods. On the neural datasets, this allows better prediction of behavioural measurements and activity in other brain areas from spiking data.\n\nSTRENGTHS:\nThe suggested method adds nicely both to the subspace identification literature and a growing body of literature in computational neuroscience that is interested in identifying shared subspaces between multivariate signals. The insight that in a linear dynamical system $(x,r,z)$ structured as in equation (7), the shared dynamics $A_{11}$ are identifiable from the time-lagged cross-covariances between $r$ and $z$ is really nice.\n\nThe core derivations of the suggested method appear solid and the experiments convincingly show that the method works (which I think for most parts is all it needs to do, as for generalized-linear time-series, to my knowledge there is little direct competition). \n\nThe manuscript is written both enjoyable and succinct. In particular the methods section -- including corresponding supplementary information -- is clear and well-written (minor comments on clarity of the results section below).\n\nWEAKNESSES:\nAs the authors remark themselves, the research question addressed here is primarily interesting for neuroscientific applications, and may hence appear somewhat niche for a larger machine learning conference. This is not a strong reservation though, in particular given the roots of the NeurIPS conferences. The contribution to the subspace identification literature should make it worthwhile to a larger audience.\n\nI am not sure if I can agree with the WLOG statement in Def 3.1 and section A.2 as it is currently written. The generality claim appears to rest on the possibility of $n_2 = n_3 = 0$ and hence $n_x = n_1$ - but in this case, what is the point of this work? I feel that the authors first should introduce the structure of shared and private latent spaces (which is not yet established by the preceding section 3.1) with the respective dimensionality split $n_x = n_1 + n_2 + n_3$. Then, given that assumption of the structured latent space with fixed $(n_1, n_2, n_3)$, equation (7) would be general.\nThe assumption A.1 is important for the algorithm and should be stated in the main text, ideally in definition 3.1 or in a small remark following it.\n\nThe multi-stage nature of the procedure feels somewhat prone to accumulation of errors -- since this is a spectral method this will hold particularly for small sample sizes. More specifically, I am a somewhat worried about numerical stability following the subtractions and subsequent matrix decompositions in equations (10) and (22) - any comments on this in the presence of estimation noise, in particular if one of the two terms shown in eq (19) to make up H dominates the other, e.g. in terms of Froebenius norm?\n\nThere is a typo in lines 822 (dropped a subscript '11' from one of the occurrences of the first block of A), and a missing subscript 'r' on matrix $C_r$ in line 967.\n\nQUESTIONS:\nI feel the first experiment (comparisons against Laplace-EM / PLDSID / bestLDS) could be a little better explained - are the comparisons trained only on the data of the primary time-series? The text only describes them as \"models of the primary time-series as learned with either Laplace-EM or [...]\". Is PSID trained on both primary and secondary time-series? A citation for Laplace-EM would be helpful. Also, what is the number of data points generated for the results in table 1? \n\nI am a little surprised about the bad performance of EM in Fig 1 compared to PLDSID - commonly, one would initialize a Poisson-EM fit with the much cheaper PLDSID fit and get at least a mild improvement. Are the shown results related to the 'Laplace'-part of the Laplace-EM algorithm? \n\nCould it be helpful to define $H$ as $H_r$ ? This notation would deviate from the classical literature only dealing with a single time series, but be consistent with the eventual definition of $H_z$ for step 3. It feels more natural to define $H$ as the Hankel matrix of the jointly-Gaussian stacked $[r, z]$, with $H_r$, $H_z$ and $H_{zr}$ collecting specific blocks.\n\nDid the matrices $Q$ described in line 998 conform with assumption A.1 ? \n\nIn A.7.1, why describe the norm as Froebenius rather than Euclidean when it's about vectors? \n\nChoosing red and green as important colors for the figures is not ideal for colorblind people.\n\nLIMITATIONS:\nThe authors discuss limitations of their work in a dedicated section.\n\nThe authors state that by the fundamental nature of their work, they see no direct positive or negative societal consequences. I tend to disagree with the authors on this point, due to the possible use of works such as this one in Brain Computer Interfaces. Those hold promise to greatly benefit people with bodily limitations such as locked-in patients.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
In this paper, the authors propose spectral learning method for learning shared latent subspaces between multiple observations. The resulting model leverages the novel construction of Hankel matrix between observations from different processes. To promote discovery of shared subspaces, the authors propose the prioritised-extension of generliased linear dynamical modeling, which prioritises explaining the primary process using the shared subspace before learning the private subspaces specific to each process (again, priority is given to the primary process). The resulting model is demonstrated on a number of neural data benchmarks.

STRENGTHS:
- The paper is clearly written, with clear pointer to mathematical details. 
- Empirical studies is comprehensive and shows convincing results.

WEAKNESSES:
- Only spectral method has been proposed for learning in the proposed multi-observation GLDM. However, one unique strength of spectral methods is that the learned dynamics given spectral methods can be used as initial parameters for EM-based methods. It would be great to see if such extension is going to further improve the capability of the model.

QUESTIONS:
See Weaknesses.

LIMITATIONS:
See Weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
This paper introduces a novel multi-step analytical subspace identification algorithm for Generalized-Linear Dynamical Models (GLDMs) to model shared and private dynamics within two time-series data sources. The proposed algorithm effectively decouples shared and private dynamics, demonstrating superior performance in simulations compared to existing methods. The algorithm's efficacy is validated using synthetic data and two non-human primate neural datasets, showcasing improved decoding accuracy of one time-series from the other.

STRENGTHS:
* The paper is well-written and has a smooth and concrete flow.
*  The paper presents a novel and effective approach to model shared and private dynamics with SSID theory.
* The experimental section is sufficient.

WEAKNESSES:
* The multi-step nature of the algorithm may introduce significant computational overhead, potentially limiting its scalability.
* The method assumes time-invariant dynamics, which may not hold for all neural data, potentially affecting its generalizability.
* While the algorithm is validated on neural datasets, its applicability to other domains or types of data is not explored, which could limit its broader impact.

QUESTIONS:
* I keep wondering what's the scientific meaning of modeling both the shared vs. private dynamics? Otherwise this would undermine the contribution of this whole work.
* How does the computational complexity of the proposed multi-step algorithm compare with existing GLDM learning algorithms, especially in terms of runtime and scalability?

LIMITATIONS:
To my knowledge and understanding, there is no potential negative societal impact of this work. Other limitations please see “Weaknesses”.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13