Disentangling the Roles of Distinct Cell Classes with Cell-Type Dynamical Systems
Authors: Aditi Jha
Diksha Gupta
Carlos D Brody
Jonathan W. Pillow
Venue: NeurIPS 2024 spotlight (2024)
Keywords: neuroscience
neural dynamics
animal decision making
Primary area: neuroscience_and_cognitive_science
TL;DR: We introduce the cell-type dynamical systems (CTDS) model, which extends latent linear dynamical systems to incorporate cell-type information when modeling neural activity.

Abstract: Latent dynamical systems have been widely used to characterize the dynamics of neural population activity in the brain. However, these models typically ignore the fact that the brain contains multiple cell types. This limits their ability to capture the functional roles of distinct cell classes, and to predict the effects of cell-specific perturbations on neural activity or behavior. To overcome these limitations, we introduce the `"cell-type dynamical systems" (CTDS) model. This model extends latent linear dynamical systems to contain distinct latent variables for each cell class, with biologically inspired constraints on both dynamics and emissions. To illustrate our approach, we consider neural recordings with distinct excitatory (E) and inhibitory (I) populations. 

The CTDS model defines separate latents for both cell types, and constrains the dynamics so that E (I) latents have a strictly positive (negative) effects on other latents. We applied CTDS to recordings from rat frontal orienting fields (FOF) and anterior dorsal striatum (ADS) during an auditory decision-making task. The model achieved higher accuracy than a standard linear dynamical system (LDS), and revealed that the animal's choice can be decoded from both E and I latents and thus is not restricted to a single cell-class. We also performed in-silico optogenetic perturbation experiments in the FOF and ADS, and found that CTDS was able to replicate the experimentally observed effects of different perturbations on behavior, whereas a standard LDS model---which does not differentiate between cell types---did not. Crucially, our model allowed us to understand the effects of these perturbations by revealing the dynamics of different cell-specific latents. Finally, CTDS can also be used to identify cell types for neurons whose class labels are unknown in electrophysiological recordings. These results illustrate the power of the CTDS model to provide more accurate and more biologically interpretable descriptions of neural population dynamics and their relationship to behavior.

Disentangling the Roles of Distinct Cell Classes with Cell-Type Dynamical Systems
Aditi Jha Electrical and Computer Engineering
Diksha Gupta Sainsbury Wellcome Center University College London diksha.gupta @ @ucl.ac.uk
Princeton University aditi jditihha@@pinneton edu
Carlos D. Brody
Jonathan W. Pillow
Princeton Neuroscience Institute
Princeton Neuroscience Institute Princeton University pillox@princeton.. edu
Princeton University brody@princeton.. edu
Abstract
Latent dynamical systems have been widely used to characterize the dynamics neural population activity in the brain. However, these models typically ignore the fact that the brain contains multiple cell types. This limits their ability to capture the functional roles of distinct cell classes, and to predict the effects of cell-specific perturbations on neural activity or behavior. To overcome these limitations, we introduce the "cell-type dynamical systems" (CTDS) model. This model extends latent linear dynamical systems to contain distinct latent variables for each cell class, with biologically inspired constraints on both dynamics and emissions. To illustrate our approach, we consider neural recordings with distinct excitatory (E) and inhibitory (I) populations. The CTDS model defines separate latents for both cell types, and constrains the dynamics so that E (I) latents have strictly positive (negative) effects on other latents. We applied CTDS to recordings from rat frontal orienting fields (FOF) and anterior dorsal striatum (ADS) during auditory decision- maaiin task. The model achieved higher accuracy than a standard linear dynamical system (LDS), and revealed that the animal's ''s choice can be decoded from both E and latents and thus is not restricted to a single cell-class. We also performed in-silico optogenetic perturbation experiments in the FOF and ADS, and found that CTDS was able to replicate the experimentally observed effects of different perturbations behavior, whereas standard LDDS model-which does not differentiate between cell types- did not. Crucially, our model allowed understand the effects of these perturbations by revealing the dynamics of different cell-specific latents. Finally, CTDS can also be used to identify cell types for neurons whose class labels unknown in electrophyssologica recordings. These results illustrate the power of the CTDS model to provide more accurate and more biologically interpretable descriptions of neural population dynamics and their relationship to behavior.
1 Introduction
Advancements in neural recording technologies have made possible to record from hundreds of neurons simultancously (32)34]. Understanding the dynamics of these high- -ith-mmennsioall populations and their relationship to complex behavior is a fundamental goal of computational neuroscience. Latent dynamical
Dynamical systems have proven be extremely useful in this pursuit 20
12]
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

systems model neural activity as arising from a low- lww-dimensionall latent state, and describe how the activity evolves as function of time in this low-dimensional space.
However, while latent dynamical systems provide parsimonious descriptions of neural activity, they fail to capture key functional and biological properties of real neural circuits. In particular, they ignore the fact that neural circuits consist of multiple cell types. Standard models describe the activity of all neurons as arising from the same set of latent variables, without any differences among cell types or constraints on the interactions between them. This prevents latent dynamical systems from shedding light on the functional roles of distinct cell classes (27 36 28 26 Furthermore, it prevents them from accurately describing optogenetic perturbation experiments, in which specific cell classes within a neural circuit are perturbed. This severely limits the usefulness of classic latent dynamical systems models for identifying the causal effects of neural activity on behavior [15] 24].
To overcome these limitations, we introduce the "cell-type dynamical systems" (CTDS) model. This model extends latent linear dynamical system (LDS) models by assigning a unique set of latent variables control the activity of each cell class, while also imposing constraints on the model parameters that dictate the sign of interactions between different cell types. Here we focus on networks with two distinct cell types: excitatory (E) and inhibitory (I) neurons. Standard analyses would characterize the dominant modes of network activity without considering the distinct roles played by and I cells. By contrast, the CTDS model defines separate latents for E and cells, and constrains the dynamics matrix so that E and latents have positive or negative (respectively) effect on the other latents. Additionally, we also constrain the mapping from the latent space to the observation space to be non-negative, thus projecting Dale's 's law from the dynamics to the observation space. Our model also easily extends to multi-region settings, and allows us to explicitly constrain the types of cells (e.g.. E cells) making long range projections.
We first demonstrate an equivalence between CTDS models with E and cells and low-rank recurrent neural networks composed of E and neurons (EI-RNN), both theoretically and using simulations. EI-RNNs 13 31 29] are widely studied as proxies of neural circuits, thus this equivalence validates the utility of our model for understanding neural circuits. Next, we apply CTDS to recordings from rat frontal orienting fields (FOF) and anterior dorsal striatum (ADS) during an auditory decision-maiee making task (4. We show that CTDS predicts neural activity better on held out trials than a standard LDS model; a multi-region CTDS with distinct latents for FOF and ADS furthermore outperforms CTDS with no regional constraints. We show that the latents extracted from both E and populations encode the animal's 's choice, revealing that choice information is not restricted to a single cell class, consistent with recent findings [23). Additionally, we show that a classifier trained on the inferred latent states during training trials predicts the animal's behavioral choice well during test trials.
Next, we use CTDS to perform in-silico optogenetic perturbation experiments modeled after those performed in vivo. In these experiments, one class of neurons (E or in a single brain region is perturbed during a portion of cach trial. Remarkably, we find that the CTDS model model-despite not being trained on perturbation data-can accurately predict the effects of different perturbations on behavior. Moreover, CTDS allows us to visualize the effects of these perturbations on the underlying dynamics of different regions, providing precise insights into the roles of distinct regions and cell-types. A standard LDS, however, is unable to replicate experimentally observed findings during the same set of perturbations due to the lack of cell information and appropriate structural constraints. Finally, we show that CTDS can be used to identify the cell class of neurons whose type is unknown in experimental recordings. While in previous analyses we assumed that the identity of neurons were known, cell-type information is often not known in neurophysiology experiments. We developed an approach to infer the identities of unknown cells using CTDS, and find that CTDS is indeed able to infer cell-types of upto 50% of unidentified neurons using recordings from the FOF and ADS. This is an exciting application of our model as tagging cell-types can be challenging during the course of real-world experiments. Overall, our findings underscore that CTDS provides versatile tool to study the effects of perturbations on neural dynamics, as well to infer cell identities.
2 Cell-type latent linear dynamical systems (CTDS)
2.1 Background: LDS models
latent linear dynamical system (LDS) model describes the firing activity of a population of N neurons at time t, y, R., as arising from a low-dimensional latent state x; E R.. This state

---

A
cell type latent linear dynamical system (CTDS)
B
latent dynamics (A)
stimulus
Ult.
Xt
Xt+11 1
excitatory
obeys Dale's law
latents
cell specific latents
inhibitory
emission (C)
latents
observed Yt
activity
time
Figure 1: Graphical model of a Cell-Type latent Dynamical System (CTDS). Red and blue circles represent the observed activity, ys, of excitatory and inhibitory neurons respectively at time f. The color the latents x, reflects the cell-type they govern. u, is any input to the system the time 1. B. Latent dynamics, A, obeys Dale's s law, such that the columns of A are either non-negative or non-positive corresponding to and latents. C. Distinct sets latents govern the activity of E and I neurons. The emission matrix, C, is also constrained be non-negative such that the Dale's law constraint in the latent space is reflected to the neural activity space.
evolves according linear dynamics and under the influence of any external input u, RM;
X++1
Ax; + Bu, + Was w, Q)
Here, A E RR DxD captures dynamics in the latent space, B Dx captures the influence of the external input, while Q E DDD is the noise covariance. The observed activity is then a linear transformation of this latent state such that
Yt Cx, + VA. V. EV
(2)
Here, E RDD called the emission matrix, and R E NNN is the observation noise covariance.
2.2 The CTDS model
To disentangle the roles of distinct cell types population, we modify the standard LDS model in two ways: (1) we define distinct set of latent variables for the activity of each cell class; and (2) we constrain the dynamics to obey functional properties of these cell types. We refer to the resulting models as Cell- Tylee latent linear Dynamical Systems (CTDS) Here we focus the setting of excitatory (E) and inhibitory (I) cell types, as schematized in Fig. We use distinct sets of latents for E and neurons, x; E Re and x; E IR D. respectively, such that D. + D; As result, the emission matrix C' is block-diagonal with blocks of shape N₂ De and N; x D:ethe first block maps the E latents to the N₂ excitatory neurons, and the second block maps the latents to the N₁ inhibitory neurons. Note that we constrain these blocks to non-negative, thus ensuring that an increase or decrease in given latent maps an increased or decreased firing rate (respectively) neurons of the associated cell type.
(shown in Fig. [C).
Next, to ensure consistency with Dale's law E neurons excite and I neurons inhibit we constrain the sign of the columns of the dynamics matrix A so that columns associated with E latents are non-negative and columns associated with latents are non-positive (Fig. BB). (Note we do not apply this constraint to the diagonal elements of A; this allows individual latents to have positive auto-correlation and evolve smoothly in time, regardless of cell type).
'Code availablehen

---

2.3 Multi-region CTDS
Given simultaneously recorded neural activity from multiple brain regions, we often want to un- derstand the computations performed by each region as well as the interactions between regions. To do so, following existing work on multi-region modeling [[2]. we extend our model multiple regions such that each region has its own set of latents. For example, if we have K regions such that x E Dt the latent vector for region ki this latent state evolves as follows:
AREX + Σ Aux
Bu: + t
jæk
where Ags E D&Dx captures the within-region dynamics of the kth region, while Azæ E Dr models the inter-region communication from region j to region k. Since we interested in cell-type specific information, we further assume separate latents for distinct cell types within a region such that x = [x k.r k. The within-region dynamics A&a obeys Dale's ' law, similar to the single-region model. Furthermore, since long range connections the cortex are generally known to excitatory 30, we can also constrain the inter-region communication matrix AJæ: to have non-negative columns corresponding the E latents of region j. and zero for the latents. Finally, to ensure that cell- type specific and region-specific latents only control their respective neurons, the emission matrix C' x is non-negative and block diagonal. Mathematicallyy we can write the activity of E and I cell classes in region k as follows:
Yt Chhi Xe
R. Vt
Vi EN( R5-0) kE ky Vt
(4)
Yt
EN(O,, R
(5)
2.4 Inference procedure
In order to infer the parameters of a CTDS model, we maximize the expected log-likelihood of observed data under the model using the Expectation Maximization (EM) algorithm (6. We will describe inference for the single-region model for simplicity, which can easily be extended to the multi-region variant. We want to infer the model parameters, θ = {A, B, C, Q, R),, given N trials of observed data {yi:r, J where n represents the trial index.
In the Expectation step, we perform standard Kalman filtering and smoothing, to learn posterior distributions of the latent states P(x" Maximization step, we learn the model parameters given the computed expectations of the latent states. Specifically, we optimize the following expression
and P(x". x yパ O) Vt (1,T,), ). In the
T Σ 1=
Th
(9) Σ
E
log.((x(+ Ax, + Bu", Q) + Σ log Nog:WWY: CX. Cx,, BI)
(6)
for (A, R). such that A and C' obey their respective structural constraints. As result, unlike a standard LDS, we no more have closed form updates for the model parameters. We instead solve two quadratic programs under the constraints on A and C' separately:
T,
log Ng( AxT + Buj 2i]
max A,B
E
s.t. A obeys Dale's law
n=1
log. No Cx2, .R)) s.t. C>= and block-diagonal
max
E
(8)
We alternate between optimizing A, B, and the noise matrices (Q, R). which have closed form expressions once {A, B, C} are fixed. Overall, alternate between the expectation and maximiza- tion steps until the log-likelihood converges. We use the MOSEK solver [2) with CVXPY [ to perform the quadratic optimizations for the constrained matrices {A, CJ.
3 Relationship to E-I recurrent neural networks
Recurrent neural networks (RNNs) are widely used to study neural dynamies in the brain 21 251 g [3. 31). An important line of work has focused RNNs with excitatory and inhibitory units,

---

which mirror the functional organization of real neural circuits [10] 13 24 31 29]. However, such models are often hard to analyze or interpret. Here we show that CTDS provides natural bridge between EI-RNNs and latent dynamical systems, in which neural activity is explained in terms of a low-dimensional latent variable. Our work thus also provides an explicit connection to low-rank RNNs, which have been shown to account for wide variety of tasks and datasets [22] g [5], but which have not to our knowledge been equipped with distinct cell types. Specifically, we show that under certain conditions a low-rank EI-RNN is formally equivalent to CTDS, and we derive the mapping from one model class to the other.
Let yt E R N be the activity generated by a linear RNN containing N neurons, such that:
Y++1 = Jyr + Tt. Te N0 P),
(9)
where J E RN-N obeys Dale's law, with non-negative columns for E cells and non-positive columns for cells, and P E R-VxN is the noise covariance. Let's further assume Ne number of E cells, and N; number of cells. For simplicity, we will focus here on the noiseless case, P O; we discuss the noisy case in the supplement (sec. AT]
We can further write the activity of the E and I cells separately as follows:
(10) where Jon and Jie represent blocks of J that contain outgoing weights from E cells and are thus non-negative, Jn and Joi are blocks of outgoing weights from cells and contain non-positive elements. Let's define new matrix J+ which contains the absolute values of J such that:
yi+1 Jeey? + Jeayi. y'+1 Jayt Jiey.
Jee Jic
J+ =
(11)
J
Here, J and J contain absolute values of the all-negative matrices Ja and Jie, respectively. Mapping this RNÑ to CTDS with E and I cells requires that the top and bottom sub-matrices of J have low-rank non-negative matrix factorization (NNMF) solutions:
Jee
J
-
[Jee
J = V2V
(12)
where U1 E RK Vi NxK, and U2 E NIKKs, V/2 E NxKa, Thus, I+ can be written as:
UV = U1
(13)
U2
V2
where U, V (a are non-negative matrices. If make the rows of V corresponding to I cells negative, and call this new matrix Vaale. we obtain J = =VV dale This EI-RNN can then be
equivalently be written as a CTDS of rank at most K₁ + K₂ as follows 35
X2 Volo Ux-1
(14)
y. UX,,
(15)
with the emissions = U, and latent dynamics constraints: C is non-negative and block diagonal, and A obeys Dale's law. Hence, in summary, a linear EI-RNN can be mapped to a CTDS if the excitatory and inhibitory rows of the absolute connectivity matrix, J+, have low-rank NNMF solutions. In the case of non-zero noise in the EI-RNN, it is still possible to find a mapping to a cell-type LDS model, however that requires further constraints on the connectivity and noise structure of the RNN (see sec. AT.
ViTie U. Both C and A obey the required
We also generated activity from E-I networks whose connectivity obeys the above constraints, and show that CTDS recovers the connectivity accurately- wwth smaller error than a standard LDS using the same amount of data (see sec. A2 Finally, when fitting CTDS models to real data, we use insights from this equivalence to initialize them. In particular, first learn a J matrix that obeys Dale's law by solving a contrained regression problem given the data {Yir)
Yt+1 Jy: + Vii
S.t J obeys Dale's law
(16)
We apply NNMF as discussed earlier on the absolute values of the learned per eq. 13 and thus initialize C = U, A V.
5

---

4 Application to rodent decision-making data
Next, we applied the CTDS model to neural data collected from rats trained to perform an auditory sreision-makinge task On each trial, animals heard randomly timed clicks from the left and the right, and were rewarded for selecting the side with more clicks (Fig. 2 Neuropixel probes were used to record simultaneous neural activity from two regions known to be involved in evidence accumulation; the frontal orienting fields (FOF) and the anterior dorsal striatum (ADS) [4 33]. Previous work has shown that individual neurons FOF and ADS exhibit side selectivity this task, with activity ramping upwards for stimuli on the cell's preferred side [14 (37. However, to carefully understand the causal role of each region during evidence accumulation (e.g.. whether regions carry out different aspects of the task), it is important to disentangle the roles of different cell types in each region. We point out that prior work has analyzed the two regions independently, while here we also study communication between them.
4.1 Model fitting details
We fitted a standard LDS and the CTDS model to zero-centered firing rates of 109 active neurons recorded in FOF and ADS regions of single rat. These neurons were filtered to have a minimum firing rate of 1Hz during the (~1s) stimulus period. We used 50ms bins to obtain firing rates from the spiking activity of neurons, and zero-centered the response of each neuron across trials. We had access to 353 trials for this animal. To fit CTDS models, we labeled neurons as E or I using both anatomical information about the two regions as well clustering of spike width histograms (see sec. A3 Additionally, since we had data from two regions, we used region identity and also fit multi-region CTDS models. Thus, we also added region-specific constraints on the dynamics matrix wwwhin--eeeieeeeeeeee regiion dynamics obeyed Dale's 's law. Since ADS is a sub-cortical region and rross-region
of this the pathway from ADS to FOF is multi-synaptic, we did not put any constraints communication from ADS to FOF, However, we restricted communication from FOF to ADS to be excitatory (as discussed in subsec. 2.3
We used 80% of 353 trials to fit our models, the remaining 20% trials were held out. The models also received 2-dimensional input at every time point, containing the number of left and right clicks played between the previous and current time points. We initialized CTDS models using the NNMF initialization process described in sec. 3 For all models, we used 10 initialization seeds and picked the best seed based on training log- likelihood. Finally, we varied the number of dimensions in the latent space of the models, and for simplicity used the same number of latents for each cell type (note that ADS has inhibitory neurons only, so we only used latents for this region when fitting CTDS models).
4.2 Both cell types encode choice information
We found that CTDS and its multi-region variant outperformed LDS models in log-likelihode of held-out trials. Fig. 2B shows test log- likeeloldood to each cell-type. This result confirms that adding E-I structure to latent linear dynamical systems allows the model to capture neural data well, while also providing additional interpretability. For each model class, we also trained a logistic regression classifier to predict the animal's choice from the last time-step of the latent state. Fig. 2C shows the choice prediction accuracy of the classifier corresponding to each model on test trials (for D = 6 per population). We observe that all models able predict choice well (and far above chance, i.e. 50%).
function of the number of latents available
Next, we analyzed a fitted multi-region CTDS model to draw scientific insights about the roles of the two regions and cell types. We chose the model with 6 dimensions per cell-type for interpretability. resulting in overall 18 latents across the two regions and cell-types (E and I for FOF, for ADS). In we show the recovered dynamics matrix- our results reveal cross-region communications between FOF and ADS in both directions and suggesting that the regions are recurrently connected during evidence accumulation tasks.
Fig. DD
Finally, we plotted the top two PCs of the inferred latent states for FOF and ADS in Fig. E with left and right choice trials colored distinctly. We found that both E and I latents in the FOF encoded the animal's choice, with their trajectories separating out for left and right choices in opposite directions. This is consistent with recent work from Najafi et al. [23], which showed that both excitatory and

---

auditory meaiiign-making task
test log- log-likelihood
choice accuracy
-648.2
test
LDS
0.5
-648.6
mult-region CTDS
(Brunton al. 2013)
2
10
multi mulliregion CTDS
dims per population
D
dynamics (A)
FOF; latents
FOF; latents
ADS; latents
right
FOF
ADS
PC1
Figure 2: CTDS applied to ciciision-making task in rodents. A. "Poisson clicks" auditory decision- making task [4] B. Log-likelihood on held out trials as function of the number of latents per cell-type, for three different model classes. L.S (gray), CTDS with no region structure (dashed black), multi-region CTDS (solid black). C. Choice accuracy (between 0-1) test trials using classifiers trained on latents inferred from the three models (with six latents per cell-type). Dotted line represents chance performance. Error bars show one standard deviation across 10 different sampled latent states. D. Recovered dynamics matrix, A, using multi-region CTDS with 6 latents per cell-type. The within-regine dynamics matrix in both regions (top left: FOF, bottom right: ADS, note that ADS has inhibitory neurons) obey Dale's s law, with columns being either excitatory only inhibitory only (we zeroed out the diagonal for visualization). E. Latent state trajectories plotted along their top two PCs colored distinctly for left (coral) and right (teal) choice trials. (Left) E and I latents in FOF, and (right) latents in ADS. The trajectories are well-separated for left and right trials.
inhibitory neurons are equally selective to choice. We also find that ADS latents encode choice information. Thus, CTDS enabled us disentangle and understand the dynamics underlying distinct classes of cells and regions.
4.3 In-silico optogenetic perturbation experiments
By dissociating dynamics underlying each cell-type, CTDS models allow us to study cell-specific optogenetic perturbations. These experiments involve activating silencing targeted class of neurons in the brain to causally establish links between neural circuits and observed behavior. We can perform in-silico perturbation experiments in CTDS models by perturbing latents corresponding to particular cell class, and consequently study the effects of these perturbations on both neural population dynamics and behavior of the animal.
During the clicks task, previous work by Hanks et al. 14] has shown that silencing excitatory neurons in FOF during the the first half of the trial does not affect the animal's behavior. However inactivation during the second half produces biased choices and reduced task accuracy. Furthermore, Yartsev et al. 37 have shown that inactivating inhibitory ADS neurons results in behavior deficits during both early and late halves of the trial.
To investigate the circuit-level origins of these effects, conducted in-silico perturbations using multi-region CTDS model from Fig. [ (with latent variables per cell type), which was modeled after invoivo perturbations. Importantly, the model was trained only on unperturbed data and did not see perturbed trials during training.
We provided the model with inputs representing left and right clicks per time bin, generated using Poisson processes. Next, we simulated perturbed data by clamping the cell-specific latents in either region to negative value during the first second half of the trial. This effectively suppressed the activity the corresponding cell class (the strength of the inactivation was determined through parameter search, see Sec. A4). In total, we generated 200 perturbed trials with equal left and right
7

---

A
silencing FOF
B
silencing ADS
12
12
model bias (%)
LDS
50
CTDS
50
bias (%)
data
0
0
start
end
start
end
Figure 3: In-silico perturbations using fitted multi-region CTDS and LDS models. A. The right y-axis displays the experimentally observed ipsilateral bias from in-vivo perturbations performed by [14] The left y-axis shows the behavioral bias predicted by the CTDS and LDS models when silencing FOF E neurons during the early and late halves of trial, relative the correct choice for that trial. For the LDS model, 50% of FOF latents were randomly silenced, as this model does not distinguish cell types. Similar A, but with ADS neurons silenced during the early and late halves of trials. Separate y-axes are used because the magnitude of model inactivation was not tuned to match the experimental bias, though the relative bias changes are well captured by the CTDS model. Error bars on the left y-axes represent one standard error across 10 different sampled latents from the fitted models, while those on the right y y-axis show one standard error across all animals.
A
unperturbed latents B silencing FOF early C silencing FOF late
latent states
FOF
-55
-55
0
ADS latent states
101
101
101
-001
-101
-101
time (s)
time (s)
time (s)
Figure Latent states of fitted CTDS during FOF perturbations. A. (Top) aveal-ae unperturbed FOF latents, with E latents in red and I latents in blue. (Bottom) Trial-averaged unperturbed ADS latents. B. (Top) FOF latents and (bottom) ADS latents, both trial-averaged, when FOF E latents silenced during the early half of trials. C. (Top) FOF latents and (bottom) ADS latents, both trial-averaged, with FOF E latents silenced during the later half of trials.
click conditions. We then predicted the choices on these perturbed trials using the classifier that had been previously trained on the unperturbed data.
Remarkably, we found that the perturbed CTDS model produced the same pattern of deficits FOF as observed in previous experimental work [[gg [A). Early-half perturbations produced low bias in both the CTDS model and in real animal experiments. On the contrary, late-half perturbations indeed resulted in biased animal behavior, with high ipsilateral bias reported from both the model and the experimental results [14. We also performed in-silico perturbations in the ADS by inactivating I latents during either early half or later half of the trial. Excitingly, we were again able to replicate experimental findings Fig. 3). Behavioral bias was high when inactivating ADS during both early half and late half, suggesting that animal behavior changes due to perturbations during both halves of a trial. These results demonstrate that the CTDS model successfully replicates the findings from optogenetic inactivation experiments.
8

---

A
unperturbed latents B
silencing ADS early C silencing FOF late
latent states
132
132
132
FOF
-132
-192
-132
0
ADS latent states
71
71
71
-71%
time (s)
time (s)
time (s)
Figure Latent states of fitted CTDS during ADS perturbations. (Top) Tvial-averaged unperturbed FOF latents, with E latents in red and I latents in blue. (Bottom) Trial-averaged unperturbed ADS latents. B. (Top) FOF latents and (bottom) ADS latents, both trial-averaged. when ADS latents are silenced during the early half of trials. C. (Top) FOF latents and (bottom) ADS latents, both trial-averaged, with ADS silenced during the later half of trials.
This motivated us to further use CTDS to visualize changes in the underlying dynamics and to understand the differential effeets of inactivations in FOF and ADS, We visualized the unperturbed latent states averaged across trials, and compared them with the perturbed latent states during the early and late phases of the same trials. We see Fig. 4B (top) that FOF latents recovered quickly after inactivations during early half of the trial (presumably using information from ADS, Fig. 4 (bottom)), and as a result we did not observe behavior deficits in this case. However, silencing FOF during later half of the trial caused the latent states of FOF to shrink, resulting in the observed behavioral deficits. Next, when perturbing ADS (Fig. BB&C), the latent dynamics changed significantly during both early and late stage inactivations explaining the behavioral deficits observed during inactivation in ADS. Thus, these results suggest that ADS is crucially involved during the task throughout, and perturbing this region results in loss of choice information in both regions, ultimately resulting in biased animal behavior.
For comparison, also trained multi-region LDS model on the same dataset, again with D 6 latents per region. This model had no explicit representation of cell-types, but it did contain distinct latents for each brain region. We attempted to replicate the perturbation experiments in this model by clamping three of the six latents in each region to negative value (just as we did with CTDS, although in this case the latents had explicit assignment to distinct cell types). As before, then used the perturbed model activity to predict choice using a classifier trained on unperturbed trials. This model accurately captured behavior on unperturbed trials (Fig. C shows error between model predictions and animal behavior). However, it notably failed to capture the pattern of deficits observed in perturbed trials (Fig. AA&B). In particular, perturbations in ADS during both halves of trial resulted in small behavioral bias, similar to early half FOF perturbations (also see subsec. [A5 for latent state visualizations). This is distinct from experimental results, which observed biased behavior during ADS perturbations but not during early-half FOF perturbations. This shows that standard LDS did not accurately capture the functional contributions of different cell-types to neural population dynamics. These results underscore the importance of cell-type models for understanding perturbation experiments, and for uncovering the contributions of different cell types population dynamics.
5 Inferring cell-type information of unknown neurons
Classifying neurons into cell-types facilitates scientific discovery, provides a nuanced view of neural circuits engaged in tasks, and is also crucial for analyses of diseases [38] However, identifying cell-types is challenging. For example, spike width is commonly used to classify putative excitatory or inhibitory neurons, but many neurons exhibit intermediate spike widths, and thus cannot be

---

A cell-type identification
B
accuracy of cell-types
Figure 6: Cell-type identification. (A) Schematic of unknown cell type prob- lem. (B) Accuracy of cell-type infer- ence as function of the number of masked neurons. Error bars show one standard error over 10 initialization seeds. Dotted line represents random performance.
1.0
frac. correct
0.5
10
30 40
50
#
masked neurons (of 109)
confidently assigned to one class or the other. Previous work has used linear dynamical systems to cluster neurons based on cell-identity [5] T6]. Here we describe method for using the CTDS model to infer cell-types from neural activity for subset of neurons in recording.
Our proposed method involves first fitting the CTDS model using only neurons with known cell type. Then, we run an altered version of the M-step, in which we select for each neuron which set of latents best describe its activity (e.g., E latents or latents). Specifically, for unknown k-th neuron, assuming diagonal observation noise R, we solve the following two regression problems:
min Eyy v
min
Ely
O* -v s.t C* >=
(17)
Here C* E IR where D is the latent dimensionality of each cell-type. This step is repeated within each M step during fitting for each unknown neuron. We then pick the Che that resulted in minimum regression error in the end, thus obtaining the identity of the neuron.
To test method, FOF-ADS data, and inferred their identities while fitting a multi-region CTDS (D = 6 latents per cell-type). Fig. 6 shows that we were indeed able infer cell identities well above chance, thus demonstrating the usefulness of CTDS for inferring of cell identities.
masked between 5 50 neurons (equal number of E and I cells) from the
Conclusions and discussion
We have developed a novel framework for disentangling the roles of distinct cell-types in neural circuits. In particular, we extended linear dynamical systems to incorporate cell-type specific information. We focused on excitatory and inhibitory cell classes, with latents constrained to interact in accordance with Dale's 's law.
We have also derived a theoretical equivalence between linear RNNs with an E-I structure and our model. Cell-type dynamical systems thus bridge gap between interpretable state-space models and mechanistic models of recurrent computation. We extended CTDS to incorporate multiple regions, resulting in multi-region CTDS models. Application of our model decision-making data from rodents revealed that including E-I structure in latent linear dynamical systems improved their ability to capture neural activity, and allowed to understand roles of distinct cell classes, In particular, in line with Najafi et al. 23 we found that both E and I neurons encode choice information.
Crucially, CTDS allowed to replicate optogenetic perturbation experiments in the FOF and ADS [4,37] due to the separation of latents based on cell types. Our model predicted the same changes in behavior as observed in experimental studies, and also allowed to visualize and understand the effects of these perturbation on the dynamics of different cell-typess. This is particularly exciting as a standard LDS unable capture the same causal effects. We also developed approach using CTDS to infer the cell-type information of neurons using their activity.
Our current model assumes linear dynamics, however the core idea of separating latents into cell types is applicable to non-linear systems as well studies should explore this extension to better capture the complexity of neural systems. Additionally, our model focuses primarily on populations containing two broad cell classes: excitatory and inhibitory neurons. However, neural populations can be subdivided into more fine-gn ane-grained cell types, and we aim for our model to be applicable to studying these more specific cell types in the future.
and should be explored in future work. Future
Overall, we believe that CTDS can be broadly useful for several applications, and can serve helpful tool to obtain a nuanced view of the dynamies underlying different populations of neurons in the brain.
10

---

7 Acknowledgements
We thank Anushri Arora, Benjamin Cowley, Lenca Cuturela and Yoel Sanchez Araujo for useful discussions and feedback at various points in the project. We thank the anonymous NeurlPS reviewers for their insightful feedback and helpful suggestions for improvement. AJ was supported by the Google PhD fellowship. JWP was supported by grants from the Simons Collaboration the Global Brain (SCGB AWD543027), the NIH BRAIN initiative (9R0IDA056404-04) an NIH RO1 (IRO1EY0S3064) and U19 NIH-NINDS BRAIN Initiative Award (U19NS104648).
References
[1] A. Agrawal, R. Verschueren, S. Diamond, and S. Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
[2] M. ApS. MOSEK version 10.1, 2024. URL BtRpsttps:/// wwt.httsss//ww..sekk.k
[3] E. W. Archer, U. Koster, J. W. Pillow, and J. H. Macke. low-dimensional models of neural population activity in sensory cortical circuits. In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. WRLLhttts///pproceedii
neurips 16w3f caw.aaaeaaaaaaaaaaaaaaaaaa..aaaa Abstract. tttstrract.htmll
[4] B. W. Brumton, M. M. Botvinick, and C. D. Brody. Rats and humans can optimally accumulate evidence for decision-making Science (New York, N.Y.), 340(6128):95-988 Apr. 2013. ISSN 1095-9203. doi: 10.1 126/science. 123rcn2cee.13312222 1233912.
[5] L. Buesing, T. A. Machado, J. P. Cunningham, and L. Paninski. Clustered factor analysis of multineuronal spike data. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL httpp:::/pprocceeinnngeeeoe http://cceeiiinnnnnnnneeeeeeeeee neurips cc/paper.pppp..c.ppppp files/ filesppppr//p/e/////eepeea /aper/ pdf
[6] A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):
1 1-22, Sept. 1977. ISSN 0035-9246, 2517-6161. doi: 101111//251--6661.177.b111..
[7] S. Diamond and S. Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1-5, 2016
[8] A. Dubreuil, A. Valente, M. Beiran, F. Mastrogiuseppe, and S. Ostojic. The role of population structure in computations through neural dynamics. Nature Neuroscience, 25(6))783-794, June 2022. ISSN 1546-1726. doi: 1111138841139--222-000533
[9] Dudman and C. R. Gerfen. Chapter 17
The Basal Ganglia, page 391-440,
Academic Press, San Diego, Jan. 2015. ISBN 978-0-12-374245-2. doi: 10. 1016/B978-0- 01-7788000122244424--0000077 attcclee/pi11B888813334444442803
http:tts////w...ceeeeeerrere cwm/sciem/eneeeeeeeeoeeeccceeeee sci ence/
[10] D. Fisher, Olasagasti, D. W. Tank, E. R. Aksay, and M. Goldman. A modeling framework for deriving the structural and functional architecture of short-term memory microcircuit. Neuron, (985),987-,000 Sept. 2013. ISSN 0896-6273, doi: 10. 1016666..euron.2000 20000000.00000..000 04..
[11] Y. Gao, E. W. Archer, L. Paninski, and J. P. Cunningham. Linear dynamical neural population models through nonlinear embeddings. In Advances in Neural Information Processing Sys- tems, volume 29, Curran Associates, Inc.. 2016. htRpsttps: /tttps::/pprocee //ptoeeddg//ooosiiiggg neur www.iinnnn.nnuireeeeeeee.er.e. cc/
Abstract...bttt.......m.c....... html
[12] J. Glaser, M. Whiteway, J. P. Cunningham, L. Paninski, and S. Linderman. Recurrent switching dynamical systems models for multiple interacting neural populations. Ad- vances in Neural Information Processing Systems, volume 33, page 14867-14878. Curran Associates, Inc., 2020. URL httpp://prooeeeeinggss
www.:eeurri.cccceeeeeee httpgggrerrrrrisssc/a/e/////0000 Ba115F777277220047766e66666666 htt.:e/eeeabbaaaaaatatt..tt Abstract.html
11

---

[13] A. Haber and E. Schneidman. The computational and learning benefits of daleian neural networks. Oct. 2022. URL http:////ppppaaroeeoooooooooorra mot//foruumiiiccccixxxZz
[14] T. D. Hanks, C. D. Kopec, B. W. Brunton, C. A. Duan, J. C. Erlich, and C. D. Brody. Dis- tinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature, 520
(75467546):220 22222)222222222222222222 Apr. 2015. ISSN 1476-4687. doi: 10. 10.1038/nature 14066.
[15] N. Li, K. Daie, K. Svoboda, and S. Druckmann. Robust neuronal dynamics in premotor cortex during motor planning. Nature, 532(760006000)459 464, Apr 2016. ISSN 1476-4687. doi: 10. 10.1088/natree 7643.
[16] S. Linderman, R. P. Adams, and J. W. Pillow. Bayesian latent structure discovery from multi-neuron recordings. In D. Lee, M. Sugiyama, U. V. Luxburg, Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 2002-2010. Curran Associates, Inc., 2016. URL http:: nips cc.paprrrc//pp 1185-bayesian www.eattsstuuuurreddccooooo pdf
[17] S. Linderman, M. Johnson, A. Miller, R. Adams, D. Blei, and L. Paninski. Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems. In A. Singh and J. Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 914-922. PMLR, 20-22 Apr 2017. URL https: / http:::://poceeddnngg mlr, www.sppess//eee press/v54/ v54 http://indmaaaa.aaaaaa
[18] S. Linderman, A. Nichols, D. Blei, M. Zimmer, and L. Paninski. Hierarchical recurrent state space models reveal discrete and continuous dynamics of neural activity in c. elegans. page 621540, Apr. 2019. doi: 10.1 101//2//24 URL httph:ttp///ste www.biioriii..oooom org/ http:x/ww..rrg/neee/////// content/10 1101/621540v1
[19] S. Linderman, B. Antin, and Zoltowski. SSM: Bayesian learning and inference for state space models, 2020. URL http: http::://thht..... com/ ss ssiindemmann..... ssm
[20] J. H. Macke, L. Buesing, J. P. Cunningham, B. M. Yu, K. V. Shenoy, and M. Sahani. Empirical models of spiking in neural populations. In Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011. URL https: h/tppers.//ppppp...ppepppp nips wttpseer.ppppppppeeeeeeeeee files/ htt
[21] V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome. Context-dependent computation by recurrent dynamies in prefrontal cortex. Nature, 503(7474))78 78-84, Nov. 2013. ISSN 1476-4687. doi: 10. 1038/natrre 2742.
[22] F. Mastrogiuseppe and S. Ostojic. Linking connectivity, dynamics, and computations in low- rank recurrent neural networks. Neuron, 99(3).609-623.c2) Aug. 2018. ISSN 08966273. doi: 10. 10.1016/j. 00.1016/j.neuron.222 neuron.201 www..2008.....000 8.07.003.
[23] S. F. Najafi, G. F. Elsayed, R. Cao, F. Pnevmatikakis, P. F. Latham, J. P. Cunningham, and A. K. Churchland. Excitatory and inhibitory subnetworks are equally selective during decision- making and emerge simultaneously during learning. Neuron, (05()):65-117.cc Jan 2020.
ISSN 0896-6273. doi:
[24] D. J. Ohea,, L. Duncker, W. Goo, X. Sun, S. Vyas, E. M. Trautmann, I. Diester, C. Ramakrish- nan, K. Deisseroth, M. Sahani, and K. V. Shenoy. Direct neural perturbations reveal dynamical mechanism for robust computation. page 16.5520767, Dec 2022.
[25] C. Pandarinath, D. J. O' O'ea,,, J. Collins, R. Jozefowicz, S. D. Stavisky, J. C. Kao, E. M. Trautmann, M. T. Kaufman, S. Ryu, L. Hochberg. M. Henderson, K. V. Shenoy, L. Abbott, and D. Sussillo. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature Methods, 851005805-8155 Oct. 2018. ISSN 1548-7105. doi: 10310 111592-018-010--)
[26] W. F. Podlaski and C. K. Machens. Approximating nonlinear functions with latent boundaries in low-rank excitatoryinniibioryy spiking networks. Neural Computation, 36(5):803-857, Apr. 2024. ISSN 0899-7667. doi: 10. 01658.
12

---

[27] J. P. Roach, A. K. Churchland, and T. A. Engel. Choice selective inhibition drives stability and competition in decision circuits. Nature Communications, 14(11):147, Jan. 2023. ISSN 2041-1723. doi: 10. 0001038//41144--23-33555
[28] S. Sanghavi and K. Kar. Distinct roles of putative excitatory and inhibitory neurons in the macaque inferior temporal cortex in core object recognition behavior, page 0023.08.01.551553
Aug. 2023. doi: 10.1101/2023,
biorxiv www.lvvv.r.oooooooo/eoe httg/contet////// htLLhttp:://met www.iiiwwwwwiiio.ooooooooooo..ia
10.1101/2023.08 05110022550000011155999999
[29] M. Schimel. Models of neural circuits as optimally driven dynamical systems. PhD thesis, Apollo University of Cambridge Repository, 2024. URL http:tttt///w.rrposssstt ac. ac.uk/handle/ 10100///6111011/184
cam
[30] B. Schuman, S. Dellal, A. Prönneke, R. Machold, and B. Rudy. Neocortical layer 1: An elegant solution to top-down and bottom-up integration. Annual Review of Neuroscience. 44(Volume 44, (221):211-222 July 2021. ISSN 0147-006X, 1545-4126. doi: 10.1146/ annurev-neuro- 100520-012117.
[31] Y. Shao and S. Ostojic. Relating local connectivity and global dynamics in recurrent excitatory- inhibitory networks. PLOS Computational Biology, 19(1)=e010885, Jan. 2023. ISSN 1553- 7358. doi: 10. 10.1371/joumaa..pct p.b.mmoa...p.i.011555......5 1010855.
[32] N. J. Sofroniew, D. Flickinger, J. King, and K. Svoboda. A large field of view two-photon mesoscope with subcellular resolution for in vivo imaging. eLife, 5:e14472, June 2016. ISSN 2050-084X. doi: 10.7554/cLife. 14422.
[33] A. Song, A. S. Charles, S. A. Koay. J. L. Gauthier, S. Y. Thiberge, J. W. Pillow, and D. W. Tank. Volumetric two-photon imaging of neurons using stereoscopy (vtwins). Nature Methods, 14(4): 420 426, Apr. 2017. ISSN 1548-7105. doi: 10. 1038/mmeth.44220
[34] N. A. Steinmetz, C. Aydin, A. Lebedeva, M. Okun, M. Pachitariu, M. Bauza, M. Beau, J. Bhagat, Kostadi- nov, C. Mora-Lopez, J. O' O'Callaghan, J. Park, J. Putzeys, B. Sauerbrei, R. J. J. van Daal, A. Z. Vollan, S. Wang. M. Welkenhuysen, Z. Ye, J. T. Dudman, B. Dutta, A. W. Hantman, K. D. Harris, A. K. Lee, E. I. Moser, J. Oeeefe, A. Renart, K. Svoboda, M. Häusser, S. Haesler, M. Carandini, and T. D. Harris. Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings. Science (New York, N.Y.), (4546) Apr. 2021. ISSN 1095-9203 doi: 10. (tl.1///ccenne.aaf.....
C. Böhm, M. Broux, S. Chen, J. Colonell, R. J. Gardner, Karsh, F. Kloosterman,
[35] A. Valente, S. Ostojic, and J. W. Pillow. Probing the relationship between latent linear dynamical systems and low-rank recurrent neural network models. Neural Computation, 341)):871-1192. 08 2022. ISSN 0899-7667. doi: 10. 10.1162/neco_a. a_002_22 URL https:://ooii 1062/
neco_ a 01522
[36] C. Whittington, W. Dorrell, S. Ganguli, and T. Behrens. Disentanglement with biological constraints: A theory of functional cell types. Feb. 2023. URL StLLhttps://ooerriiiee net/ Siaamidd=92_Gfhinne
[37] M. M. Yartsev, T. D. Hanks, A. M. Yoon, and C. D. Brody. Causal contribution and dynamical encoding in the striatum during evidence accumulation. eLife, 7:e34929, Aug. 2018. ISSN 2050-084X. doi: 10.7554/cLife.3499
[38] H. Zeng and R. Sanes. Neuronal cell-type classification: challenges, opportunities and the path forward. Nature Reviews Neuroscience, 1809):530-546, Sept. 2017. ISSN 1471-0048. doi: 10. 1038//m..011..85.
13

---

A Appendix
A1 Relationship between noisy EI-RNN and CTDS
In sec. 3
wc showed that in the case of no noise, is possible to map a linear RNN with E-I structure CTDS under certain conditions on the connectivity of the RNN. Here, discuss the conditions under which a noisy EI-RNN maps to a CTDS.
Following sec.
let us assume the following linear RNN with N neurons:
(A1) We know from sec. 3 that for this RNN to map to CTDS, J = UVT, where U and V are low rank matrices such that U is non-negative block diagonal and obeys Dale's law. As result, in the noiseless case, the CTDS that captures this model will have emission C U, and the dynamics
yt+1 Jy: + me m ())
In the noisy case, let's look at the joint distribution of the first two consecutive observations in a linear RNN model:
y1
N
Jyo (1/50)
[ PJJ
JP JPJ +P))
(A2)
Y2
Next, we can also write the joint distribution of observations in any LDS model: CAQUT
([ [CCA)
[CQCT + CQACT CAQATC" CQCT + R
Y1 y2
-
-CQAC
(A3)
Now, if C U and A VTU, the means of the observations in both settings match already. However, we need to compute the noise terms Q and R, so that the resultant CTDS maps to a noisy EI-RNN.
Following the joint distributions of observations in an RNN and an LDS, following three expres- sions should hold true:
P= ==CQCT + R
(A4)
JP = CAQC
(A5)
JPJ + == CCAAAAA CT + +QQC +
(A6)
(A7)
Let's start with the second expression:
JP CAQCT Since C = and VTU, we obtain:
JP -V-UVTUL UUUUUTUQU
(A8)
UV P-UVT
(A9)
Q-UtPU
(A10)
Since U is low-rank, not all of the covariance of P will be captured by Q. Thus, R should capture the remaining noise:
R= -1 R=
(A11)
Thus, JP = CAQCT is now satisfied.
However, we also want:
P= CQC +R= +R
(A12)
Thus, for linear RNN be perfectly captured by CTDS, the eigenvectors of P should either be aligned to the space spanned by U or to the space orthogonal to it.
Finally, the third remaining equation has stricter implications:
JPJT = CAQATC
(A13)
(UVT) UV uQu
(A14)
14

---

connectivity matrix
error J
El RNN
example generated activity
2.0
RMSE
recovered
0
CTDS
LDS
Figure A1: Simulations with an E-Inororrk A Schematic of an E-I RNN, with red units representing excitatory cells and blue units representing inhibitory cells. Example generated activity from 10 randomly selected neurons. B. Top shows the true connectivity matrix J, while the bottom plot shows the recovered using a CTDS. C. Root mean squared error between true and recovered J using standard LDS and CTDS model, each of latent state dimensionality 4. We fitted them to data from two E-I networks with 100 and 200 units each. The shaded region represents 95% confidence interval.
UQUT is low-rank and thus cannot capture P entirely-it only captures covariance in the subspace spanned by U. Hence, for the LHS and RHS to be equal, we need U and VT to have aligned subspaces so that the effect of projecting P on V results in the same covariance as that on the RHS.
So. in summary, we are able to find a restricted linear low-rank RNN that can be perfectly captured by be expressible as UVT, with rank D. Here, V
CTDS. Specifically, we want the connectivity should be in the column-space of U and obey Dale's 's law, while U should be positive block diagonal. Finally, we require that the noise covariance has eigenvectors either completely aligned with the U subspace or entirely orthogonal to this subspace. In such a case, we have an CTDS that perfectly captures the activity the RNN with following parameters:
C U, A=VTU
(A15)
Q = PUIT
(A16)
R (I - -U- (( (U--UUTI--UU -0000-00UJU---0000000
(A17)
A2 Simulations with an E-IRNN
In order to illustrate the mapping between an EI-RNN and CTDS, we generated simulated activity from two E-I networks with 100 and 200 units respectively. The connectivity matrix, J, had non-negative rank of for each of the sub-matrices formed using the excitatory and inhibitory rows. In each case, the connectivity matrix J was constrained to obey Dale's law, and was normalized to have eigenvalues less than 1. We set the noise matrix P in accordance with sec. A1
We then generated 10 trials of 1000 time steps each from each network (Fig. Al A), and fitted both standard LDS and CTDS models of latent-space dimensionality 4 to the generated activity.
We initialized both models randomly, and computed recovered connectivity matrices post-fitting as follows:
= C'EE C +R)
(A 18)
We solve the æyapunov equation E, AE,-AA + obtain E As shown in Fig. C, we found that the CTDS model recovers the connectivity matrix in both settings accurately, while standard LDS does much worse in terms of root mean squared error. Fig. AlB shows the true and recovered connectivity matrices.
A3 Assigning cell-types to neurons in FOF and ADS
In this section, we discuss how we assigned cell-types to neurons ADS and FOF. The ADS part of the striatum which is known to have primarily inhibitory neurons 99) hence assumed
15

---

mean spike widths for FOF cells
number of calls
Figure A2: Histogram of the spike widths of FOF neurons. We see bimodal distribution.
mean spike width (ms)
A
unperturbed latents
B
silencing ADS early C silencing ADS late
latent states
FOF
335
-35
-35
latent states
50
50
50
ADS
-50
-50
-50
time (s)
time (s)
time (s)
Figure A3: Latent states of fitted LDS during ADS perturbations. A. (Top) Trial-averaged unperturbed FOF latents, with randomly selected E (red) and (blue) latents. (Bottom) Tveeleaee unperturbed ADS latents. B. (Top) FOF latents and (bottom) ADS latents, both trial-averaged. when ADS latents are silenced during the early half of trials. C. (Top) FOF latents and (bottom) ADS latents, both trial-averaged, with ADS silenced during the later half of trials.
all ADS neurons be I cells. To identify cell types in the FOF, we constructed histogram of the average spike widths of neurons in FOF. We found this to be a bimodal distribution (Fig. and thus we labeled the neurons that had spike width less than 0.4ms as inhibitory and the remaining as excitatory. This is also consistent with the known distribution of neuronal cell types in this region (20% inhibitory, 80% excitatory).
A4 Inactivation strength during in-silico perturbation experiments
As discussed in 4.3 we performed in-silico optogenetic perturbation experiments in fitted CTDS model by clamping latents corresponding to E. cells in FOF, and those corresponding to cells in ADS. In order to choose the inactivation strength for clamping the latent states, varied the magnitude of the latents between [-1, 10) during the perturbation time steps (either the first half of trial, or the later half of trial). Since the model was on neural data that was processed to have mean firing rate of OHz for every neuron, to ensure that clamping of latents indeed resulted in the inactivation of the corresponding neurons, we wanted their median firing rate be negative. Hence, we chose the inactivation magnitude that resulted in a median firing rate of at least - -HHz across all neurons being perturbed. This resulted in an inactivation strength 2 FOF E latents, and for ADS I latents (Fig. 3 shows results for these values). We did not tune the inactivation strength to match the ipsilateral hias from the experimental works [44 inactivation techniques.
as different studies used distinct
16

---

A5 LDS Latent space visualization during in-silico perturbations
Finally, we also visualized the latent trajectories of standard multi-region LDS model when performing in-silico perturbations (Fig. A3 Here, we inactivated ADS latents during either early half or late half of trials. While in a multi-region CTDS model, inactivations in ADS derailed the latent dynamics entirely (Fig. we found that in a standard LDS the latent trajectories recovered quickly after perturbations during both halves of trials (Fig. A3). This is likely because CTDS constrained the connectivity between neurons and regions based on structural constraints in the brain, while the LDS did not have any such constraints. effects of perturbation experiments, while LDS is unable to provide an accurate representation of underlying brain dynamics.
result, CTDS able to accurately capture the dynamical
A6 Compute requirements and code
Finally, we trained these models on 2.8 GHz Intel Cascade Lake with 8 CPU cores, and developed our code on top of [19..
17

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have elaborated on all claims made in the abstract and introduction in the main paper, and they reflect the paper's contributions accurately.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in the paper and in the discussion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
We do not have any theorems or lemmas, as this is not a theory paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have provided details of all parameters, hyperparameters and other model- ing details in the paper and the supplement combined.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [No]
Justification: We release code for the model fitting, the github link is in the manuscript. However, the experimental data used cannot be released at this time.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https i
//nips //nipscc//pie wccppublccgguuuiecceeeeeee.oo.ee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, provide all details in the paper and supplement.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We have strived to add error bars in all our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes].
Justification:
We mention this the appendix of our paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
This paper adheres to the NeurIPS code of conduct.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
We do not envision any societal impact of this work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The answer NA means that the paper does not existing assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "I5MNy4JVQM", "reviewer": "Reviewer_yzj8", "summary": "This work describes how a latent dynamical systems (LDS) model of neural activity can incorporate distinct excitatory (E) and inhibitory (I) latents. Doing so requires being careful to maintain sign constraints (Dale's law) in the transition matrix defining latent dynamics as well as the emission matrix defining how recorded neural activity is a projection of the underlying latents. This model can also be extended to a multi-area variant, enabling separate interaction between and within areas. The model, named a \"cell-type dynamical system\" (CTDS), can be fit to neural activity using an expectation-maximization inference procedure.\n\nThe model is extensively evaluated, and shown to be better than vanilla LDS. On neuropixel data from a rodent auditory decision-making task, the model predicts neural activity and behavioral choice better. Simulated optogenetic perturbations on the model better predicts behavioral effects of real perturbations. Finally, the model can be used to infer putative E/I identities of recorded neurons.", "questions": "See above.", "limitations": "I didn't see limitations in the Discussion section as mentioned in the Checklist.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "**Significance**: Latent dynamical systems are a widespread model used to explain neural activity, and this work takes the necessary step of incorporating E/I information to make these models more biologically realistic and closer to a mechanistic model. As such, this work tackles an important problem for the neuroscience community. The model is also simple and elegant, which should help its adoption.\n\n**Novelty**: The model seems novel enough, though sign constraints is a common feature of computational neuroscience models, so it's surprising if such an approach hasn't been explored. (Can the authors better contextualize this in the Introduction?) Nevertheless, even if that were the case, such convergence is appreciated.\n\n**Technical Quality**: The work is technically solid, with an impressively thorough evaluation on simulated data / RNN theory, predictions for neural/behavioral data, perturbations for behavior effects, and inference of cell types.\n\n**Presentation Quality**: The writing, the mathematical notation, and the figures are all very well done.", "weaknesses": "1. **Cell types**: The name \"cell-type dynamical systems\" seems to be overselling a bit. It could more accurately be called \"sign-constrained latent dynamical systems\" or \"latent dynamical systems with Dale's Law\". Of course, the notion of \"cell types\" (e.g. Pyramidal, SST, PV, VIP, Purkinje) includes the excitatory/inhibitory distinction, but this is just one of many functional properties (morphology, intrinsic dynamics, stereotyped connectivity, plasticity rules, etc.) that are widely accepted as important to the distinction of cell types. Crucially, while it's relatively straightforward to incorporate sign constraints into the LDS framework as demonstrated in this work, it is not obvious to me how to incorporate these other properties that characterize cell types. If the authors feel that CTDS can indeed easily capture such properties, I would be interested to see a discussion here and in the paper about how such extensions can be done.\n\n2. **Limitations**: I didn't see limitations in the Discussion section as mentioned in the Checklist. Please include a separate subsection to discuss this explicitly. In particular, I'd be interested to understand: (a) How the dimensionalities of the latent population are chosen for different tasks, (b) How this framework can scale to more difficult and naturalistic tasks than 2AFC.\n\n3. **Presentation tweaks**: A couple minor revisions to improve the manuscript. Equation 1 and 3 should use a similar notation/definition for the error term. Figures use should use the same hues/shades of red and blue to denote excitatory and inhibitory data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed  and insightful review. We are grateful that the reviewer found our work to be novel, technically solid, and of significance to the neuroscience community. Below are our responses to the reviewer’s comments and questions:\n\n1. **Relevant literature on sign constraints in neuroscience models**:  We agree with the reviewer that a large number of previous models have sought to incorporate Dale’s law and or sign-constrained weights.  However, we would like to point out that past works have focused on incorporating Dale’s law in recurrent neural network models (RNNs), but NOT (as far as we are aware) into latent dynamical system models. For example, [Fisher et al. 2013, Haber & Schneidman 2022] have used sign-constrained RNNs to study properties of neural circuits, and [O’Shea & Duncker et al., 2022] have used them to study perturbation experiments . We have cited these works in Sec 3 of our paper, but will also mention them in the introduction for better contextualization per the reviewer’s suggestion.\nHowever, we are not aware of similar literature in the context of latent dynamical systems. Latent dynamical systems have typically focused on descriptive representations of neural activity, while sidestepping mechanistic properties of the brain. Our work, thus, bridges this gap by enforcing mechanistic constraints on linear dynamical systems. We thank the reviewer for bringing this up, and we will make this clear in the introduction of our revised manuscript.\n\n\n2. **Extension to other cell types, model nomenclature:** We agree with the reviewer that here we only focus on E and I cell types, and have not discussed more fine-grain cell identities.  However, we feel that our framework can easily incorporate multiple cell types within a single population or across multiple populations (as discussed in the example case of PV and SOM inhibitory neurons in the response to all reviewers). We apologize for not making this generality clear in our original paper, and will revise both the Introduction and Discussion sections to discuss specific ways in which CTDS can be applied to multiple cell types (and not just E and I). \n\n\n3. **Limitations section:** Thanks for pointing this out. We will add an explicit limitations section in the manuscript discussing choice of latent dimension, and scaling to other tasks. Currently, we choose latent space dimensionality based on test log-likelihood as well as for ease of visualization / interpretation. However, our results qualitatively hold for a wide range of latent dimensions (we will add this in the appendix). \nIn terms of applying the model to more naturalistic tasks beyond 2AFC, two main challenges would come up:\n (a) The stimulus could be high-dimensional: this would mean learning a correspondingly higher-dimensional B matrix in our current CTDS setup. Alternatively, we could also learn a lower-dimensional encoding of the stimulus, depending on the specifics of the task.\n (b) For naturalistic tasks spanning over larger timescales, a linear model might not be sufficient. In such cases, our approach can be extended to incorporate sign constraints in a non-linear setup such as a switching linear dynamical system.\n\n\n4. **Presentation tweaks**: Thank you pointing these out, we have fixed these now.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c5BTSQeKeE", "reviewer": "Reviewer_KBww", "summary": "This work proposed the dynamical model with cell-type specific latent, especially focused on the excitatory (E) and inhibitory (I) neurons. They developed a cell-type dynamical system (CTDS), where E/I neurons will only have positive or negative effects. They apply this model into decision making takes, and CTDS outperforms standard LDS model in decoding animal choice. They also performed an in silico experiment with optogenetics stimulation to test the causal effects on behaviors. In the end, they also demonstrate CTDS could help with identifying cell types.", "questions": "1. What might be the potential challenges to extend this framework into nonlinear model?\n2. How to extend this into more fine-grained cell type levels? subclass (vip, sst, pvalb, etc.)", "limitations": "No potential negative societal impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Motivation**\n\n1. Neurons with different cell types has very distinguishable roles in neural computation, while it is often ignored in dynamical modeling for neural activities. This paper focused on important problem, and model cell types with their postive/negative effects, and demonstrating the effectiveness of the proposed model.\n\n**Results**\n\n1. They evaluated their proposed models on extensive applications, including decoding choices, studying causal effects from in silico experiments, and inferring cell types.\n2. The results on studying causal effects are very impressive, given the model is only trained on unperturbed data, that demonstrated the learned model is very generalizable to new scenarios.", "weaknesses": "**Evaluation**\n\nThe proposed model only compared with over-simplified linear RNNs model, adding nonlinearity to model and add more powerful baselines will add more strengths into the evaluations. \n\n**Results**\n\nThe CTDS does not outperform simple LDS in several settings, i.e. Fig 2c, Fig 3c.\n\n**Applications**\n\nKnowing cell types information in advance could be difficult in many experiment setups, using anatomical information or spike width histograms could add errors into cell type information. Evaluating the robustness of the results given incorrect cell type information. \n\n**Related works**\n\nRelevant references on modeling cell-type specific neural dynamics and inferring cell types is missing.\n\n[1] Learning Time-Invariant Representations for Individual Neurons from Population Dynamics. NeurIPS 2023.\n\n[2] Transcriptomic cell type structures in vivo neuronal activity across multiple timescales. Cell Report 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive assessment of our paper and constructive suggestions for improving it. We are glad that the reviewer found our experiments to be extensive, and our causal perturbation results to be impressive. However, we feel that the reviewer has misunderstood one of our key findings, as we explain below. We apologize for not making this point clearer, and would humbly request that the reviewer consider raising their score if indeed they are persuaded by our response (on that and several other issues that we describe below).\n\n1. **Comparison between CTDS and a standard LDS (Fig 2C, 3C):** We would like to clarify the takeaway messages from both these figures, and want to emphasize that they do not suggest that LDS models outperform CTDS. Fig 2C shows a comparison between choice accuracy as decoded from CTDS and LDS models. Indeed, both models perform equally well, however CTDS provides greater interpretability by allowing us to characterize the information encoded by the different  cell types. Furthermore, CTDS has higher test-LL which means it in fact captures neural activity __better__ than LDS.  \nMore importantly, in Fig 3C (left panel), we show the classification error when perturbing FOF during either half of a trial. Experimental perturbation results from rats show that this error is low during early-half perturbations, but high during late-half perturbations (see Fig 3C right panel). CTDS is indeed able to capture this phenomenon due to the distinction between E and I latents, while a regular LDS does __not__ recapitulate these findings. The fact that the “choice error” bar for the LDS model is lower than that of the CTDS model in Fig 3C does not mean that the LDS model is performing better — in fact, the opposite is true, since the point of this figure is to show that CTDS model captures the qualitative pattern of errors found in real behavior experiments (from Hanks et al. 2015, shown in the right panel of Fig 3C) better than LDS. We apologize for the confusion on this point, and we will rewrite the figure caption and text to make this clear. This is a key contribution of our work, so we hope this alleviates the reviewer’s concern.\n\n\n2. **Comparisons to nonlinear RNNs, incorporating non-linearity:** Our rationale for showing an equivalence between CTDS and linear RNNs was to illustrate that CTDS acts as a bridge between mechanistic and descriptive models. That is, low-rank linear E-I RNNs can be exactly transformed to a CTDS model.  We agree with the reviewer, however, that non-linear RNNs are more expressive. In future work we would certainly like to extend CTDS to incorporate nonlinearities, which we agree would make it both more expressive and more biologically plausible; we will certainly add this point to the Discussion.  However, we feel that the idea of incorporating cell types into LDS models, along with the empirical results on perturbation experiments (and the  mathematical connection to linear RNNs) nevertheless make for a worthwhile contribution in their own right, which we hope the reviewer will agree on.\n\n\n3. **Evaluating the robustness of the model using incorrect cell-type information:** This is a great question, and we thank the reviewer for bringing this up. As the reviewer points out, it may be difficult to brain accurate cell-type information. Sec 5 of our paper which focuses on inferring cell type identities gets at this question: we mask the identities of up to 50% neurons in our dataset, and fit CTDS to these datasets while also inferring neuron identity. As shown in Fig 4, CTDS is able to infer neuron identity significantly above chance for up to 50% of neurons. To answer the reviewer’s question more directly, we have also attached a figure showing test log-likelihood of CTDS fitted to datasets with varying numbers of masked neurons, as compared to when we know all cell identities accurately (in the pdf attached with author rebuttal). As we can see, while test LL expectedly falls off as the percentage of masked neurons increases, it is robust when masking up to 20% of neurons. We will add this new result to our revised manuscript.\n\n4. **Relevant literature on inferring cell classes:** Thank you for pointing us to these papers, we will be sure to cite and discuss them in the updated version of the manuscript.\n\n5. **Adding more fine-grained cell information:** This is an important suggestion. Please see response to all reviewers for our comments on this.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MjYFZyPyuT", "reviewer": "Reviewer_upoR", "summary": "The author developed a model named Cell-Type Dynamical Systems (CTDS) to capture the excitatory (E) and inhibitory (I) neurons’ electrical activity of rat frontal orienting fields (FOF) and anterior dorsal striatum (ADS) during an auditory decision-making task.", "questions": "The author focuses on two neuron cell types: excitatory (E) and inhibitory (I) neurons. Is it possible to extend it to other neuron cell types?\nHow does the CTDS model perform on neural recordings from other brain regions and tasks? \nIs the CTDS model feasible for real-time applications, such as closed-loop experiments where neural activity needs to be decoded and perturbed in real time?\nFrom figure 2E, the author stated that ‘both E and I latents in the FOF encoded the animal’s choice, with their trajectories separating out for left and right choices in opposite directions’. Does this mean only I latent of the ADS region participating in encoding the animal’s choice? But from Figure 2D, these two regions are recurrently connected. Then, what is the function of the E neurons in ADS during the decision-making process?", "limitations": "Addressed limitation.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The experiments are well-designed and comprehensive. The results are well presented. The author systematically explores the CTDS models’ performance on E and I neurons’ electrical activity as well as these two neurons’ activity within two brain regions. The model shows higher accuracy in predicting neural activity compared to standard LDS models. By separating latent for E and I neurons, the model provides deeper insights into the functional roles of different cell classes.", "weaknesses": "1.\tIt is hard to interpret the Figure 2D; what is the y-axis of the plot for eigenvalues? \n2.\tFor Figure 2E, why is the ADS, E latents not plotted?\n3.\tWhat is the dashed line in the ipsilateral bias plot in Figure 3CD? What does the star mean in the plot, especially in Figure 3D?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive assessment of our work. We are glad that the reviewer found our work to be well presented, and our experiments to be comprehensive. Below are out detailed responses to the reviewers questions:\n\n1. **Clarifications on figures 2D / 3CD:** Thanks for bringing these to our attention, we will add detailed captions to clarify these figures. In Fig 2D, the y-axis represents the magnitude of the eigenvalues. In Fig 3CD, the dashed line shows 0% bias, which means that the animal makes perfect choices. In Fig 3D, the stars indicate that the experimentally observed ipsilateral biases during both early-half and later-half perturbations in the ADS are significantly above 0% (Yartsev et al. 2018). In Fig 3C, the observed bias is only significant during later-half FOF perturbations as indicated by the star, and not during the early-half perturbations. Furthermore, the difference between the two perturbation effects in FOF is significant (Hanks et al. 2015). \n\n\n2. **E latents in ADS:** Thanks for pointing this out. ADS does not have E neurons, thus we thought it was not important to visualize E latents in ADS. We discuss this in sec A3 of the appendix, but will make sure to mention this in the main text to avoid confusion. \n\n\n3. **Extension to other cell types:** Thanks also for this comment; we feel this is an important point about the generality of our model, and we have elaborated on this in more detail in our response to all reviewers.\n\n\n4. **Performance on other brain regions and tasks:** We focused on one task in this work, and fitted our models to one dataset containing FOF and ADS neurons. However, we would like to point out that our results (Fig 3CD) are consistent with two other papers (Hanks et al. 2015 and Yartsev et al. 2018), which study the same task but using distinct datasets. Our focus here is to introduce the model, and comprehensively demonstrate its merits using the poisson clicks task. We hope future work will apply CTDS to other tasks and brain regions.\n\n\n5. **Real-time decoding and perturbations:** CTDS is indeed capable of real-time decoding. Once the model has been fitted, during inference Kalman filtering can be used to infer underlying latent states and behavior. Thus, it is also possible to extend this framework to control the inputs being provided to the model in a closed loop while running real-time decoding. Thanks for this important comment, we will add this to the discussion section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their positive assessment of our work, and for their detailed comments and suggestions. We are delighted that the reviewers found our work to be well-written, of significance to the community, and our experiments to be well-designed and technically solid. We thank reviewers yzj8 and upoR for placing our work above the acceptance threshold. We also thank reviewer KBww for their constructive feedback; we have done our best to address their concerns in our detailed rebuttal below, and would humbly request that they consider raising their score.

We will discuss a general point mentioned by all reviewers first, and then address reviewer-specific comments.

**Incorporating additional cell types beyond E and I**: As the reviewers have pointed out, our paper focuses primarily on populations with two cell classes: excitatory and inhibitory neurons. Indeed, neural populations can be divided into more fine-grain cell-types, and we intend for our model to be applicable to these cell-types. For example, for a population with two distinct classes of inhibitory neurons (e.g., PV and SOM), we could use the CTDS modeling framework to incorporate distinct latent variables for each of these two populations, and the dynamics matrix could be structured to assume for particular forms of connectivity between these populations as determined by anatomy (e.g., that both PV and SOM cell types inhibit excitatory neurons, and that SOM neurons inhibit PV, but not vice versa).  However, we realize that we did not make this point clear in our original submission, and we will clarify the generality of our framework in the Introduction and Discussion sections of the revised manuscript.  

Note that we have already considered a case with two different regions, where the projection from one region to the other is purely excitatory, whereas the other allows connections of both signs (once again, as suggested by anatomy).  We therefore prefer to keep the name “Cell Type Dynamical Systems” for our framework, although we are open to additional suggestions if the reviewers feel strongly.  We hope that future work can use our approach to disentangle information encoded by a range of different cell types.

Meta-review: This paper proposes a novel latent dynamical system model for neural data that can distinguish between cell types. The reviewers unanimously agree that the paper is innovative and technically solid and therefore suitable for NeurIPS. However, two reviewers raised concerns that, although the title and abstract broadly refer to cell types, the proposed method mainly concerns excitatory and inhibitory cells. It remains unclear whether the method would generalize to other cell types. Aside from this, based on my reading of the discussions, most of the reviewers' comments have either been addressed or are not seen as barriers to the paper's acceptance at NeurIPS.
Decision: Accept (spotlight)