Title: Uncovering motifs of concurrent signaling across multiple neuronal populations
Venue: NeurIPS 2023
Accepted: yes
Keywords: neuroscience

multi-population neural recordings

dimensionality reduction

latent variable models

Gaussian processes
Abstract: Modern recording techniques now allow us to record from distinct neuronal populations in different brain networks. However, especially as we consider multiple (more than two) populations, new conceptual and statistical frameworks are needed to characterize the multi-dimensional, concurrent flow of signals among these populations. Here, we develop a dimensionality reduction framework that determines (1) the subset of populations described by each latent dimension, (2) the direction of signal flow among those populations, and (3) how those signals evolve over time within and across experimental trials. We illustrate these features in simulation, and further validate the method by applying it to previously studied recordings from neuronal populations in macaque visual areas V1 and V2. Then we study interactions across select laminar compartments of areas V1, V2, and V3d, recorded simultaneously with multiple Neuropixels probes. Our approach uncovered signatures of selective communication across these three areas that related to their retinotopic alignment. This work advances the study of concurrent signaling across multiple neuronal populations.

Uncovering motifs of concurrent signaling across multiple neuronal populations

Abstract

Modern recording techniques now allow us to record from distinct neuronal populations in different brain networks. However, especially as we consider multiple (more than two) populations, new conceptual and statistical frameworks are needed to characterize the multi-dimensional, concurrent flow of signals among these populations. Here, we develop a dimensionality reduction framework that determines (1) the subset of populations described by each latent dimension, (2) the direction of signal flow among those populations, and (3) how those signals evolve over time within and across experimental trials. We illustrate these features in simulation, and further validate the method by applying it to previously studied recordings from neuronal populations in macaque visual areas V1 and V2. Then we study interactions across select laminar compartments of areas V1, V2, and V3d, recorded simultaneously with multiple Neuropixels probes. Our approach uncovered signatures of selective communication across these three areas that related to their retinotopic alignment. This work advances the study of concurrent signaling across multiple neuronal populations.

1 Introduction

Cortical circuits functionally involve feedforward, feedback, and horizontal interactions between many neuronal populations that span distinct areas and layers. Recording techniques now allow us to record from many neurons across these populations [1-3] (Fig. 1a). To capitalize on the scientific opportunities presented by these recordings, however, new conceptual and statistical frameworks are needed, particularly as we consider communication across multiple (more than two) populations.

Characterizing interactions between just two populations is a challenging high-dimensional problem. Dimensionality reduction techniques have therefore been increasingly used for this purpose [4-7]. Methods like canonical correlation analysis (CCA) [8] and its probabilistic variants [9], in particular, identify a low-dimensional set of latent variables that parsimoniously describe the interactions between two populations. This cross-population shared-latent model has inspired several extensions targeted toward neural recordings [10-14].

Communication between two populations, however, occurs bidirectionally and likely concurrently, and disentangling this concurrent communication is a substantial challenge in neuroscience. A

a

b

A

B

A

B

A2

Neuron B1

B2

Time

Activity

pattern

Time

Neurons

Trials

c

Two distinct interactions?

One global interaction?

Neural

activity

Latent

activity

A

B

C

A

B

C

A

B

C

Population A

Population B

recently developed dimensionality reduction approach, delayed latents across groups (DLAG) [15], addresses this challenge by leveraging two insights: (1) communication between two populations is not instantaneous, and (2) while concurrently relayed signals might be difficult to tease apart from the raw neural activity (Fig. 1b, black trajectories in each population space), they could be pinpointed along certain dimensions or axes (Fig. 1b, 'latent activity' measured along the magenta and gray dimensions; note how activity in one population leads activity in the other population).

Pairwise methods are ill-equipped, however, to analyze communication across three or more recorded populations. Suppose, for example, that we wish to study the interactions of three recorded populations, A, B, and C (Fig. 1c). One could consider applying one of the aforementioned methods to each pair of populations. However, we would encounter the following interpretational ambiguity. Suppose that populations A and B exhibit shared activity fluctuations, and populations A and C also exhibit shared fluctuations. Do populations A, B, and C all co-fluctuate together (Fig. 1c, right; black arrows: direction of influence)? Or do A and B co-fluctuate in a way that is uncorrelated with the way in which A and C co-fluctuate (Fig. 1c, left)? Only by analyzing all populations together can we differentiate these possibilities. Multi-population dimensionality reduction approaches, such as group factor analysis (GFA) [16], could be applied toward that end, but the challenge of disentangling concurrent, bidirectional signaling remains. New approaches are needed to jointly characterize the multi-dimensional, concurrent flow of signals among multiple populations.

We therefore propose multi-population DLAG (mDLAG), a dimensionality reduction framework that determines (1) the subset of populations described by each latent dimension, (2) the direction of signal flow among those populations, and (3) how those signals evolve over time within and across experimental trials. We illustrate these features in simulation, and further validate mDLAG by applying it to previously studied recordings from neuronal populations in macaque visual areas V1 and V2. Then we study interactions across select laminar compartments of areas V1, V2, and V3d, recorded simultaneously with multiple Neuropixels probes. mDLAG uncovered signatures of selective communication across these three areas that related to their retinotopic alignment. Throughout the analyses of simulated data and neural recordings, mDLAG provided improved interpretation and quantitative performance relative to alternative approaches. This work advances the study of concurrent signaling across multiple neuronal populations and its role in brain function.

2 Delayed latents across multiple groups (mDLAG)

Observation model and automatic relevance determination For population m (comprising q m neurons) at time t on trial n , we define a linear relationship between observed activity, y m n,t ∈ R q m , and latent variables, x m n,t ∈ R p (Fig. 2a):

a

c

N

trials

M

populations

0

1

A    B

B    A

t

2

-

t

1

0

0

1

GP auto-

covariance

GP cross-

covariance

b

Latent 1

7

A

B

C

GP parameters

Automatic relevance

determination

where the loading matrix C m ∈ R q m × p and mean parameter d m ∈ R q m are model parameters. 1 ε m is a zero-mean Gaussian random variable with noise precision matrix Φ m ∈ S q m × q m ( S q m × q m is the set of q m × q m symmetric matrices). As we will describe, at time point t , latent variables x m n,t , m = 1 , . . . , M are coupled across populations, and thus each population has the same number of latents, p . Because we seek a low-dimensional description of neural activity, the number of latent variables is less than the number of neurons, i.e., p < q , where q = ∑ m q m .

A core goal of the mDLAG framework is to identify and distinguish multiple network-level interactions (Fig. 1c). To do so requires identifying the number of latent variables across all recorded populations, and which subset of populations each latent involves, in a computationally tractable manner. We take a Bayesian approach to this problem (thereby avoiding computationally intensive grid search, see Discussion), and let d m , Φ m , and C m be probabilistic parameters with prior distributions.

The parameter d m describes the mean firing rate of each neuron. We set a Gaussian prior over d m :

where β ∈ R > 0 is a hyperparameter, and I q m is the q m × q m identity matrix. We constrain the precision matrix Φ m = diag ( ϕ m 1 , . . . , ϕ m q m ) to be diagonal to capture variance that is independent to each neuron. This constraint encourages the latent variables to explain as much of the shared variance among neurons as possible. We set the conjugate Gamma prior over each ϕ i m , for each neuron i = 1 , . . . , q m :

where a ϕ , b ϕ ∈ R > 0 are hyperparameters.

The loading matrix C m linearly combines latent variables and maps them to observed neural activity. In particular, the j th column of C m , c j m ∈ R q m , maps the j th latent variable x m n,j,t to population m . To determine which subset of populations is described by each latent, we employ automatic relevance determination (ARD), which has been used successfully in a variety of contexts [17, 18]. Specifically, we define the following prior over the columns of each C m :

where α j m ∈ R > 0 is the ARD parameter for latent variable j and population m , and a α , b α ∈ R > 0 are hyperparameters. As α j m becomes large, the magnitude of c j m becomes concentrated around 0 , and hence the j th latent variable x m n,j,t will have a vanishing influence on population m . The ARD prior encourages population-wise sparsity for each latent variable during model fitting (see below), where the loading matrix coefficients will be pushed toward zero for latent variables that explain an insignificant amount of shared variance within a population, and remain non-zero otherwise (Fig. 2b).

The parameter C m also has an intuitive geometric interpretation. Each element of y m n,t , the activity of each neuron in population m , can be represented as an axis in a high-dimensional population activity space (Fig. 1b). Then each column of C m defines a dimension in population m 's activity space (Fig. 1b, 'activity pattern'). Dimensions that appear in two or more populations represent patterns of activity that are correlated across populations. Note that the columns of C m are linearly independent; but they are not, in general, orthogonal. The ordering of these columns, and of the corresponding latent variables, is arbitrary.

State model For each latent variable, we seek to characterize the direction of signal flow among the involved populations (determined by ARD) and how those signals evolve over time within and across trials. We therefore employ Gaussian processes (GPs) [19], and define a GP over all time points t = 1 , . . . , T for each latent variable j = 1 , . . . , p as follows (Fig. 2c):

Under equation 7 , latents are independent and identically distributed across trials. The diagonal blocks K 1 , 1 ,j = · · · = K M,M,j ∈ S T × T describe the autocovariance of each latent, and each T -byT off-diagonal block describes the cross-covariance that couples two populations.

To define these matrices, we introduce additional notation. Specifically, we indicate populations with two subscripts, m 1 = 1 , . . . , M and m 2 = 1 , . . . , M . Then, we define K m 1 ,m 2 ,j ∈ R T × T to be either the auto- or cross-covariance matrix between latent variable x m 1 n,j, : ∈ R T in population m 1 and latent variable x m 2 n,j, : ∈ R T in population m 2 on trial n . We choose to use the squared exponential function for GP covariances (Fig. 2c). Therefore, element ( t 1 , t 2 ) of each K m 1 ,m 2 ,j can be computed as follows [15, 20]:

where the characteristic timescale, τ j ∈ R > 0 , and the GP noise variance, σ 2 j ∈ (0 , 1) , are deterministic model parameters to be estimated from neural activity. δ ∆ t is the kronecker delta, which is 1 for ∆ t = 0 and 0 otherwise.

We also introduce two new parameters: the time delay to population m 1 , D m 1 ,j ∈ R , and the time delay to population m 2 , D m 2 ,j ∈ R . Notice that, when computing the auto-covariance for population m (i.e., when m 1 = m 2 = m ; Fig. 2c, top), the time delay parameters D m 1 ,j and D m 2 ,j are equal, and so ∆ t (equation 9) reduces simply to the time difference ( t 2 -t 1 ) . Time delays are therefore only relevant when computing the cross-covariance between distinct populations m 1 and m 2 . The time delay to population m 1 , D m 1 ,j , and the time delay to population m 2 , D m 2 ,j , by themselves have no physically meaningful interpretation. Their difference D m 2 ,j -D m 1 ,j , however, represents a well-defined, continuous-valued time delay from population m 1 to population m 2 (Fig. 2c, bottom). The sign of the relative time delay indicates the directionality of the lead-lag relationship between populations captured by latent variable j (positive: population m 1 leads population m 2 ; negative: population m 2 leads population m 1 ), which we interpret as a description of signal flow. Note that time delays need not be integer multiples of the sampling period or spike count bin width of the neural activity.

Without loss of generality, we designate population m = 1 as the reference area, and fix the delays for population 1 at 0 (i.e., D 1 ,j = 0 for all latent variables j = 1 , . . . , p ). We follow the same conventions as in [15, 21], and fix σ 2 j to a small value ( 10 -3 ). Furthermore, the GP is normalized so

that k m 1 ,m 2 ,j ( t 1 , t 2 ) = 1 if ∆ t = 0 , thereby removing model redundancy in the scaling of the latent variables and loading matrices C m .

Posterior inference and fitting the mDLAG model Let Y and X be collections of all observed neural activity and latent variables, respectively, across all time points and trials. Similarly, let d , ϕ , C , A , and D be collections of the mean parameters, noise precisions, loading matrices, ARD parameters, and time delays, respectively. From the neural activity, we seek to estimate posterior distributions over the probabilistic model components θ = { X, d , ϕ , C, A} and point estimates of the deterministic GP parameters Ω = { D, { τ j } p j =1 } .

We do so by employing a variational inference scheme [16, 22], in which we maximize the evidence lower bound (ELBO), L ( Q, Ω) , with respect to the approximate posterior distribution Q ( θ ) and the deterministic parameters Ω , where

We constrain Q ( θ ) so that it factorizes over the elements of θ :

This factorization enables closed-form updates during optimization. The ELBO can then be iteratively maximized via coordinate ascent of the factors of Q ( θ ) and the deterministic parameters Ω . Here all hyperparameters were fixed to a very small value, β, a ϕ , b ϕ , a α , b α = 10 -12 , to produce noninformative priors [16]. Throughout this work, we take estimates of the latent variables and model parameters to be the corresponding means of the posterior distributions comprising equation 11. Full details are provided in Supplementary Section S2.

mDLAG special cases Finally, we consider some special cases of the mDLAG model that illustrate its relationship to other dimensionality reduction methods. In the case of two populations ( M = 2 ), mDLAG is equivalent to a Bayesian formulation of DLAG [15]. In the case of one population ( M = 1 ), and when all time delays are fixed to zero ( D m,j = 0 ), mDLAG becomes equivalent to a Bayesian formulation of Gaussian process factor analysis [23]. By removing temporal smoothing (i.e., in the limit as all GP noise parameters σ j approach 1 ) mDLAG becomes equivalent to GFA [16].

3 Validation in simulation and on neural recordings

Simulation 1: Uncovering directed interactions across multiple populations We first sought to demonstrate the ability of mDLAG to infer the structure of multi-population interactions. We therefore generated simulated neural activity from three populations ( M = 3 ) according to the mDLAG generative model (equations 1-9). For the sake of illustration, we set 10 neurons in each population ( q m = 10 ). Importantly, we designed the loading matrices C m so that all types of interpopulation interactions were represented (Fig. 3a, left, ground truth): interactions shared globally, unique to each pair of populations, and local to one population. We also scaled the observation noise precision matrices Φ m so that noise levels were representative of realistic neural activity. Specifically, activity due to single-neuron observation noise was 10 times stronger than activity due to latents (the 'signal-to-noise ratio' tr ( C m C m ⊤ ) / tr ((Φ m ) -1 ) = 0 . 1 for each population). We selected Gaussian process timescales and (relative) time delays that ranged between 20 ms to 150 ms and between 5 ms and 50 ms, respectively. With all model parameters specified, we then generated N = 100 independent and identically distributed trials. Each trial was 500 ms in length, comprising T = 25 time points with a sampling period of 20 ms, to mimick the 20 ms spike count time bins used to analyze the neural recordings in Section 4.

We then fit an mDLAG model to the simulated neural activity. We set the initial number of latents ( p = 10 ) to be greater than the ground truth number ( p = 7 ), to verify that these additional latents would be pruned during fitting. In addition, to demonstrate the benefit of ARD in the mDLAG model, we also fit a modified mDLAG model that did not use ARD. Specifically, we fit (via an exact EM algorithm) a modified model with state model defined by equations 7-9 and observation model defined by equations 1 and 2, but with no prior distributions over the parameters C m , d m , and Φ m (i.e., we obtained point estimates). The number of latents for this modified model was selected via 4-fold cross-validation, considering model candidates with p = 1 to p = 10 latents. Cross-validation is not needed for the mDLAG model with ARD, a key computational advantage of the approach.

mDLAG

(no ARD)

Latent 1

7

Ground

truth

mDLAG

(with ARD)

a

A

B

C

Estimate

Ground truth

b

A

B

C

8.0

14.0

7.6

13.6

B

A

14.0

15.4

C

A

34.0

34.4

B

C

46.0

42.9

0

2.4

-2.4

250 ms

Time

0

2.4

-2.4

A

B

C

Latent 1

2

3

4

5

6

7

We were able to identify the total number of latent variables ( p = 7 ) with both approaches. The mDLAG model with ARD additionally recovered the population-wise sparsity structure with high accuracy (Fig. 3a, center). The mDLAG model without ARD, however, produced an estimate of the loading matrix with mostly non-zero elements (Fig. 3a, right): had we not known the ground truth in advance, it would be difficult to interpret which population subsets are involved in which interactions. The mDLAG model with ARD also estimated the latent variable time courses and time delays (along with their implied signal flow) with high accuracy (Fig. 3b; R 2 between ground truth and estimated time courses: 0.936; mean delay error: 1.14 ms).

Simulation 2: Disentangling concurrent, bidirectional signaling We next considered a simulated scenario that illustrates mDLAG's ability to distinguish concurrent, bidirectional signaling across populations, particularly in contrast with static multi-population methods like group factor analysis (GFA). 2 We again simulated activity from three populations, each with q m = 10 neurons. These populations interacted through two latent processes, one that propagated from population A to B to C (Fig. 4a, Latent 1) and one that propagated in the opposite direction, from population C to B to A (Fig. 4a, Latent 2). Each latent incurred a 15 ms time delay propagating from one population to the next. For the sake of visual demonstration, we generated latent signals with a simple transient peak of activity (Fig. 4a, black traces), and generated the same latent signals on each trial. We then generated observed neural activity according to the mDLAG observation model (equations 1 and 2). Importantly, we scaled the columns of the loading matrices C m (for each area m ) so that both columns had equal magnitude, and thus both latent interactions exhibited equal strength in each population: such a communication scheme is difficult to disentangle [15]. Finally, to isolate the consequences of model structure from issues like overfitting, we simulated a low-noise, data-rich scenario by generating N = 1 , 000 independent trials, each 500 ms in length (comprising T = 25 time points), with high signal-to-noise ratios tr ( C m C m ⊤ ) / tr ((Φ m ) -1 ) = 10 . 0 for each population.

We fit both an mDLAG model and a GFA model (see Supplementary Section S3) to this simulated neural activity, assuming for each model the correct number of latent variables, p = 2 . mDLAG latent variable and time delay estimates accurately reflected the distinct signaling pathways across

2

0

2.2

-2.2

Latent 1

c

GFA estimate

a

15 ms

15 ms

15 ms

15 ms

Latent 1

Latent 2

Time

A

B

C

b

mDLAG estimate

0

2.7

-2.7

100 ms

Time

One trial

Latent 1

2

A

B

C

14.7

15.0

14.8

14.9

A

B

C

the three populations (Fig. 4b; R 2 between ground truth and estimated time courses: 0.989; mean delay error: 0.16 ms). Each latent estimated by GFA, however, notably reflected a mixture of both interactions (Fig. 4c, each latent time course exhibits two peaks). With no description of signal flow, static methods like GFA struggle to distinguish concurrent, bidirectional signaling.

Validating mDLAG on recordings from V1 and V2 To further validate mDLAG, we turned to neural recordings from two areas in the macaque visual cortex, V1 and V2 [24]. These recordings have been studied extensively [15, 25-27], and therefore provide an excellent testbed for our framework. We applied mDLAG to the simultaneously recorded activity of neuronal populations in the superficial (output) layers of V1 (61 to 122 neurons; mean 86.3), and the middle (input) layers of V2 (15 to 32 neurons; mean 19.6) in three anesthetized animals (Supplementary Fig. S1a). We analyzed neuronal responses measured during the 1.28 second presentation of drifting sinusoidal gratings of different orientations, and counted spikes in 20 ms time bins. Because we were interested in V1-V2 interactions on timescales within a trial, we subtracted the mean across time bins within each trial from each neuron. This step removed activity that fluctuated on slow timescales from one stimulus presentation to the next [28]. In total, we analyzed separately 40 'datasets,' corresponding to five recording sessions, each with eight different orientations. Each dataset included 400 trials: we allocated at random 300 trials as a training set and 100 trials as a test set on which to measure model performance (we employed a leave-group-out prediction metric; see Supplementary Section S4).

Given prior work on these V1-V2 recordings, we reasoned that mDLAG ought to qualitatively recover several hallmarks of the V1-V2 activity: (1) a significant number of latent variables local to each area, indicative of selective V1-V2 communication (Supplementary Fig. S1b, 'V1' and 'V2'; Supplementary Fig. S1c, latents 4 and 5), (2) the presence of periodic structure representative of the drifting grating stimulus (Supplementary Fig. S1c, latents 3 and 4), and (3) bidirectional interactions (Supplementary Fig. S1c, latents 1-3). Indeed, mDLAG detected all of these hallmarks.

These recordings were also previously studied with DLAG [15], which was designed to study concurrent signaling between two neuronal populations. Since DLAG is a special case of the mDLAG framework, we reasoned that mDLAG ought to exhibit similar quantitative performance and extract qualitatively similar features of V1-V2 communication. Indeed, the latent time courses estimated by mDLAG (Supplementary Fig. S1c) were qualitatively similar to those estimated by DLAG[29] (Supplementary Fig. S1d). Interestingly, mDLAG outperformed DLAG across all datasets (Supplementary Fig. S2a, points above the diagonal), suggesting that ARD provides an improved method of model selection over the constrained grid search method used for DLAG, while also avoiding grid search's computational drawbacks (see Supplementary Section S5). Collectively, the results across our simulations and the V1-V2 recordings indicate that mDLAG is well-equipped to study concurrent signaling across multiple simultaneously recorded neuronal populations.

a

Neuropixels

probes

Layer 4

V1

V2

V3d

d

c

V1

V2

V3d

2°

2°

b

Neuron 1

562

1,000

Time from stim. onset (ms)

V1

V2

V3d

S

M

D

S

M

D

S/M

0

e

No. shared dims.

5

0

Unique

Global

V1-V2

V1-V3d

V2-V3d

0

4.4

-4.4

333 ms

Time

One trial

Latent 1

2

3

f

6 ms

10 ms

4 ms

V1

V2

V3d

2 ms

V1

V3d

V1

V3d

S

M

D

V1

V2

V3d

log firing rate

-1

2

4 Interactions across laminar compartments of V1, V2, and V3d

We next used mDLAG to study interactions across V1, V2, and V3d of anesthetized macaques recorded using multiple Neuropixels probes (Fig. 5a). These recordings included hundreds of neurons across different layers in each area (Fig. 5b, superficial (S), middle (M), and deep (D) compartments) allowing us to investigate longstanding questions about functional cortical organization.

First, we sought to quantify the performance of mDLAG models relative to alternative approaches, and to further demonstrate the empirical benefit of mDLAG's model components. We therefore considered three types of models: group factor analysis (GFA); mDLAG, but with time delays removed ('mDLAG-0'); and mDLAG. With each model type, we analyzed spike counts (in 20 ms bins) of neurons in select laminar compartments of each area, measured during the first 1,000 ms after the onset of a drifting sinusoidal grating (Fig. 5b, trial-averaged responses for an example dataset). The selected laminar compartments (grouped into three sub-populations for analysis: Fig. 5c, color shading) were those that likely directly interacted through a combination of feedforward, feedback, and inter-laminar connections [30] (Fig. 5c, black arrows). As we did for the V1-V2 recordings, above, we subtracted the mean across time bins within each trial from each neuron, to remove slow fluctuations beyond the timescale of a trial. We fit GFA, mDLAG-0, and mDLAG separately to 10 datasets, comprising five recording sessions (each with two grating orientations, 90° apart, presented 300 trials each) from two anesthetized animals. For each dataset, we allocated at random 225 trials as a training set and 75 trials as a test set on which to measure model performance. GFA, mDLAG-0, and mDLAG exhibited increasingly better leave-group-out prediction (Supplementary Fig. S2b: mDLAG0 better than GFA; c: mDLAG better than mDLAG-0), demonstrating the performance benefit of (1) temporal smoothing and (2) time delays (see also Supplementary Fig. S3 for a demonstration of mDLAG performance versus number of available training trials, and Supplementary Fig. S4 for a characterization of GFA and mDLAG runtimes).

We then focused on one particular set of questions: Do V1, V2, and V3d communicate selectively, and if so, what aspects of the recorded populations might contribute to this selectivity? One likely aspect is the alignment of visual receptive fields (RFs): retinotopically aligned neuronal populations might co-contribute to computations that non-aligned populations would not. This functional organization would appear as mDLAG latent variables shared exclusively between the aligned populations. We therefore studied a dataset from a recording session in which the retinotopy of the recorded populations allowed us to investigate this hypothesis (Fig. 5d). RFs of the V1 and V3d populations were largely overlapping (Fig. 5d, blue: V1, gold: V3d), whereas the RFs in V2 did not overlap with either V1 or V3d (Fig. 5d, red: V2).

We quantified the total number of dimensions shared between each pair of populations (Fig. 5e; a dimension was considered significant in a population if it explained at least 2% of the shared variance, Supplementary Section S5, equation S52). mDLAG allowed us to distinguish dimensions that were unique to a pair of populations (e.g., between V1 and V3d, but not V2) from dimensions that also involved the third population. V1-V3d interactions included unique dimensions (Fig. 5e, 'V1-V3d', light gray) in addition to a dimension shared globally across all three populations (Fig. 5e, 'V1-V3d', dark gray), whereas V1-V2 interactions and V2-V3d interactions could be attributed primarily to the global dimension (Fig. 5e, 'V1-V2' and 'V2-V3d', only dark gray). The single-trial time courses of the global latent evolved according to a long timescale (Fig. 5f, latent 1, 132 ms) relative to the unique V1-V3d latents (Fig. 5f, latents 2 and 3, 8 ms and 62 ms, respectively). The global latent was also associated with a direction of signal flow (V3d to V2 to V1) opposite to that of the V1-V3d latents (V1 to V3d). The time courses of the V1-V3d latents exhibited features of the stimulus response, including a fast transient response (Fig. 5f, latent 2) and periodic structure with the same period as the drifting grating stimulus (Fig. 5f, latent 3, 333 ms period). Results were largely insensitive to the initialization of the mDLAG fitting procedure (Supplementary Fig. S5). These findings are consistent with the hypothesis that visual cortical populations communicate selectively in a retinotopic manner, as they perform computations on inputs from localized regions of visual space [31].

5 Discussion

Even for two populations, identifying which latents are shared between populations or local to a population is a challenging computational problem. Existing approaches explicitly designate latents as shared or local [13-15], and then rely on heuristics to avoid the computational cost of grid search. Scaling this type of approach to three or more populations (or external experimental variables) would be prohibitive. mDLAG is thus an advance toward scaling to large-scale multi-population recordings, and could continue to be improved through the many approaches used to scale Gaussian process methods in neuroscience [13, 23, 32-36].

mDLAG treats time delays as constant parameters across trials, time, and neurons. The delay estimated for each latent variable thus represents a summary of inter-population interaction throughout the course of an experiment. To assess the variability of delay estimates, one could fit mDLAG to subsets of trials, subsets of neurons, or to separate trial epochs. If additional subpopulation labels are available (for example, laminar compartments in our Neuropixels recordings, Fig. 5b), one could also specify these subpopulations to the mDLAG model to better incorporate variability of delays across neurons.

A latent variable shared across three or more populations is potentially consistent with multiple signaling schemes. For instance, Latent 1 of Fig. 5f is consistent with a signal relayed from V3d to V2 (with a delay of 2 ms) and then from V2 to V1 (with a delay of 4 ms). It is also consistent, however, with a scheme in which V3d is a common input to V2 (with a delay of 2 ms) and to V1 (with a delay of 6 ms). A third scheme could involve common input to all three areas from an unobserved source. Still, mDLAG narrows the set of populations that could be involved in any given interaction, and the sign and magnitude of mDLAG's time delays narrow the set of signaling schemes consistent with the data. This hypothesis set can be narrowed further by experimental interventions [37].

The mDLAG model includes assumptions of linearity and temporal smoothness; specifically, we have employed here the commonly used squared exponential function for mDLAG's GP covariances. It might be desirable to incorporate temporal structure more appropriate for certain signals (for example, the sinusoidal structure of Latent 3, Fig. 5f). Then, an alternative GP covariance function can be substituted in a straightforward manner into the mDLAG model (equation 8) [19, 32, 33, 38].

Nonlinearities, for example related to neuronal tuning or receptive fields, could also be incorporated under a GP framework [39, 40].

For exploratory data analysis, mDLAG's GP-based description of multi-population temporal structure is advantageous over an alternative linear dynamical system (LDS)-based description [10, 11] in two respects: (1) a GP can be useful for exploratory data analyses where an appropriate parametric dynamical model is unknown a priori , and (2) mDLAG's continuous-time model enables the discovery of wide-ranging delays with high precision, which, in contrast to discrete-time LDS approaches, are not limited to be integer multiples of the sampling period or spike count bin width of the neural activity. Ultimately, these approaches can be complementary: one can use mDLAG to generate data-driven hypotheses about motifs of concurrent signaling across populations, and then test these hypotheses with a dynamical system-based approach.

Finally, while we applied mDLAG to the spiking activity of neuronal populations, mDLAG is applicable to any high-dimensional time series data, including other neural recording modalities. In fact, the groups analyzed by mDLAG need not all be neuronal populations, but could include, for example, a collection of dynamic stimulus or behavioral variables. mDLAG is a general framework that advances the study of concurrent signaling throughout the brain.

Code availability

A MATLAB (MathWorks) implementation of mDLAG is available on GitHub at http://github. com/egokcen/mDLAG and on Zenodo at https://doi.org/10.5281/zenodo.10048163 [41].

Acknowledgments

We thank A. Zandvakili for providing the V1-V2 recordings in Supplementary Fig. S1. This work was supported by the Dowd Fellowship (E.G.), ECE Benjamin Garver Lamme/Westinghouse Fellowship (E.G.), Simons Collaboration on the Global Brain 542999 (A.K.), 543009 (C.K.M.), 543065 (B.M.Y.), NIH RF1 NS127107 (A.K., C.K.M., B.M.Y.), NIH R01 EY035896 (B.M.Y., A.K., C.K.M.), NIH U01 NS094288 (C.K.M.), NIH CRCNS R01 MH118929 (B.M.Y.), NSF NCS DRL 2124066 (B.M.Y.), NIH R01 NS129584 (B.M.Y.).

References

Ahrens, M. B. et al. Brain-wide neuronal dynamics during motor adaptation in zebrafish. Nature 485, 471-477 (2012).

Yang, W. & Yuste, R. In vivo imaging of neural activity. Nature Methods 14, 349-359 (2017).

Steinmetz, N. A. et al. Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings. Science 372, eabf4588 (2021).

Semedo, J. D., Gokcen, E., Machens, C. K., Kohn, A. & Yu, B. M. Statistical methods for dissecting interactions between brain areas. Current Opinion in Neurobiology 65, 59-69 (2020).

Kang, B. & Druckmann, S. Approaches to inferring multi-regional interactions from simultaneous population recordings. Current Opinion in Neurobiology 65, 108-119 (2020).

Keeley, S. L., Zoltowski, D. M., Aoi, M. C. & Pillow, J. W. Modeling statistical dependencies in multi-region spike train data. Current Opinion in Neurobiology 65, 194-202 (2020).

Machado, T. A., Kauvar, I. V. & Deisseroth, K. Multiregion neuronal activity: the forest and the trees. Nature Reviews Neuroscience 23, 683-704 (2022).

Zhuang, X., Yang, Z. & Cordes, D. A technical review of canonical correlation analysis for neuroscience applications. Human Brain Mapping 41, 3807-3833 (2020).

Bach, F. R. & Jordan, M. I. A Probabilistic Interpretation of Canonical Correlation Analysis. Technical Report No. 688, Department of Statistics, University of California, Berkeley (2005).

Semedo, J., Zandvakili, A., Kohn, A., Machens, C. K. & Yu, B. M. Extracting Latent Structure From Multiple Interacting Neural Populations. Advances in Neural Information Processing Systems 27, 2942-2950 (2014).

Glaser, J., Whiteway, M., Cunningham, J. P., Paninski, L. & Linderman, S. Recurrent Switching Dynamical Systems Models for Multiple Interacting Neural Populations. Advances in Neural Information Processing Systems 33, 14867-14878 (2020).

Bong, H. et al. Latent Dynamic Factor Analysis of High-Dimensional Neural Recordings. Advances in Neural Information Processing Systems 33, 16446-16456 (2020).

Keeley, S., Aoi, M., Yu, Y., Smith, S. & Pillow, J. W. Identifying signal and noise structure in neural population activity with Gaussian process factor models. Advances in Neural Information Processing Systems 33, 13795-13805 (2020).

Balzani, E., Noel, J.-P. G., Herrero-Vidal, P., Angelaki, D. E. & Savin, C. A probabilistic framework for task-aligned intra- and inter-area neural manifold estimation. The Eleventh International Conference on Learning Representations (2023).

Gokcen, E. et al. Disentangling the flow of signals between populations of neurons. Nature Computational Science 2, 512-525 (2022).

Klami, A., Virtanen, S., Leppäaho, E. & Kaski, S. Group Factor Analysis. IEEE Transactions on Neural Networks and Learning Systems 26, 2136-2147 (2015).

MacKay, D. J. Bayesian Non-Linear Modeling for the Energy Prediction Competition. ASHRAE transactions 100, 1053-1062 (1994).

Neal, R. M. Bayesian Learning for Neural Networks, PhD thesis (University of Toronto, 1995).

Rasmussen, C. E. & Williams, C. K. I. Gaussian processes for machine learning (MIT Press, Cambridge, MA, 2006).

Lakshmanan, K. C., Sadtler, P. T., Tyler-Kabara, E. C., Batista, A. P. & Yu, B. M. Extracting Low-Dimensional Latent Structure from Time Series in the Presence of Delays. Neural Computation 27, 1825-1856 (2015).

Yu, B. M. et al. Gaussian-Process Factor Analysis for Low-Dimensional Single-Trial Analysis of Neural Population Activity. Journal of Neurophysiology 102, 614-635 (2009).

Bishop, C. Variational principal components. 9th International Conference on Artificial Neural Networks, 509-514 (1999).

Jensen, K., Kao, T.-C., Stone, J. & Hennequin, G. Scalable Bayesian GPFA with automatic relevance determination and discrete noise models. Advances in Neural Information Processing Systems 34, 10613-10626 (2021).

Zandvakili, A. & Kohn, A. Paired V1-V2 neuronal spiking responses in anesthetized macaque monkey. https://doi.org/10.6080/K0B27SHN (2019).

Zandvakili, A. & Kohn, A. Coordinated Neuronal Activity Enhances Corticocortical Communication. Neuron 87, 827-839 (2015).

Semedo, J. D., Zandvakili, A., Machens, C. K., Yu, B. M. & Kohn, A. Cortical Areas Interact through a Communication Subspace. Neuron 102, 249-259 (2019).

Semedo, J. D. et al. Feedforward and feedback interactions between visual cortical areas use different population activity patterns. Nature Communications 13, 1099 (2022).

Cowley, B. R. et al. Slow Drift of Neural Activity as a Signature of Impulsivity in Macaque Visual and Prefrontal Cortex. Neuron 108, 551-567 (2020).

Gokcen, E. DLAG (v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.6654831 (2022).

Salin, P. A. & Bullier, J. Corticocortical connections in the visual system: structure and function. Physiological Reviews 75, 107-154 (1995).

Carandini, M. et al. DoWeKnowWhattheEarly Visual System Does? Journal of Neuroscience 25, 10577-10597 (2005).

Ulrich, K. R., Carlson, D. E., Dzirasa, K. & Carin, L. GP Kernels for Cross-Spectrum Analysis. Advances in Neural Information Processing Systems 28, 1999-2007 (2015).

Gallagher, N. et al. Cross-Spectral Factor Analysis. Advances in Neural Information Processing Systems 30, 6842-6852 (2017).

Zhao, Y. & Park, I. M. Variational Latent Gaussian Process for Recovering Single-Trial Dynamics from Population Spike Trains. Neural Computation 29, 1293-1316 (2017).

Duncker, L. & Sahani, M. Temporal alignment and latent Gaussian process factor inference in population spike trains. Advances in Neural Information Processing Systems 31, 10445-10455 (2018).

Dowling, M., Zhao, Y. & Park, I. M. Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains. Proceedings of the 40th International Conference on Machine Learning (2023).

Reid, A. T. et al. Advancing functional connectivity research from association to causation. Nature Neuroscience 22, 1751-1760 (2019).

Wilson, A. & Adams, R. Gaussian Process Kernels for Pattern Discovery and Extrapolation. Proceedings of the 30th International Conference on Machine Learning, 1067-1075 (2013).

Wu, A., Roy, N. A., Keeley, S. & Pillow, J. W. Gaussian process based nonlinear latent structure discovery in multivariate spike train data. Advances in Neural Information Processing Systems 30, 3496-3505 (2017).

Duncker, L., Ruda, K. M., Field, G. D. & Pillow, J. W. Scalable Variational Inference for Low-Rank Spatiotemporal Receptive Fields. Neural Computation 35, 995-1027 (2023).

Gokcen, E. mDLAG (v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.10048163 (2023).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work extends an established line of previously proposed methods for Gaussian Process factor analysis-based models of neuroscience data. In comparison with previously published methods, it extends the factor model to include correlations and time delays between different (matched) latents underlying different populations. Inference is performed via mostly closed-form variational updates. This method is applied to data from paired recordings from three regions in visual cortex, providing evidence of directional influence between them in the context of overlapping receptive fields.\n\nThis is a valuable contribution that extends previous methods in ways that are likely to prove increasingly useful in an era of widespread multi-region recordings. While the algorithmic innovation is somewhat incremental, the validation, presentation, and contextualization to questions in neuroscience are very good.\n\nSTRENGTHS:\n- This is a valuable contribution that extends previous methods in ways that are likely to prove increasingly useful in an era of widespread multi-region recordings. While the algorithmic innovation is somewhat incremental, the validation, presentation, and contextualization to questions in neuroscience are very good.\n- The manuscript is clearly and carefully written, including a helpful notation key in the supplement.\n- Rigorous model evaluation through recovery of parameters on synthetic data and careful analysis of a challenge data set.\n- Details of the model and its inference are carefully laid out in the main text and supplement.\n\nWEAKNESSES:\n- While the authors have done a commendable job of using a consistent and careful notation, the resulting formulas are index-dense, and it may be easy for some readers to miss the key conceptual setup, particularly which latents are independent of which other latents. For example, l. 94 in the supplement clearly notes that latent posterior estimates are independent for different trials, and this is implicit in (7), but this probably deserves a mention in the main text. Similarly, different $\\mathbf{x}_j$ for $j=1\\ldots p$ appear to also be independent, and this is a substantive assumption (see comment under limitations below), but I didn't see this discussed (though I may have missed it).\n\nQUESTIONS:\n- The authors report several latents with nearly sinusoidal form. Are inferred delays in these cases well-defined? Is it possible to infer directionality in these cases, when a lag of $2\\pi(1  - D/\\tau)$ (with $\\tau$ the period) should also produce similar results? This seems to me clearer in cases of sparse signals or strong transients but more difficult when latents are densely active.\n\n- While the authors cite a number of related factor analysis models in neuroscience (especially refs 14, 15), there is a limited discussion of some other lines of multi-region work (e.g., ref 33, but also Gallagher et al. NeurIPS 2017 from the same group) that have previously used a GP-based factor analysis model and included relative delays between regions. What, precisely, is the difference between these models apart from their application to spikes as opposed to LFP?\n\nLIMITATIONS:\n- From (7), it appears as if the latents for each $j=1\\ldots p$ are independent: $\\mathrm{cov}[x^m_{njt}x^m_{nj't}] \\propto \\delta_{jj'}$. That is, there is potential coupling _across populations_ for each latent but not _across latents_ even within a population. This choice might warrant some additional discussion in the text.\n- The variational inference algorithm for the latents requires inverting an $MpT \\times MpT$ matrix for each trial. I suspect this is feasible for typical numbers ($M \\sim 3$, $p \\sim 10$, $T \\sim 10$) and can be parallelized across trials, but will scale poorly if any of the relevant parameters become large.\n- While the model is fit to spike count data, the observation model in (1) is Gaussian. This is likely to hinder inference when bins are small and/or counts are low.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops a new approach to analyze multi-region neural populations recordings. This approach extends DLAG to analyze communication across more than 2 regions. The main technical novelty lies in tractably extending the model definition of DLAG to multiple regions through the incorporation of the ARD prior. I found the approach elegant and particularly relevant to the current state of neuroscience research where multi-region recordings are growing in number. DLAG and mDLAG like approaches offer the ability to temporally disentangle concurrent signals, allowing us to make sense of multi-region recordings. The experiments are thorough and all the figures are very well made and described. Overall, I very much enjoyed reading this!\n\nSTRENGTHS:\n1. The paper is extremely relevant for a computational neuroscience crowd, where analyzing multi-region recordings is an important challenge. This approach can in practice help drive neuroscience research and answer questions about signaling across regions in the brain.\n2. Technically, the novelty lies in the model description (which allows for easy disentangling of latents involved across region(s)) and in discovering a tractable inference scheme (I should say that I did not read details of the inference in the supplement).\n3. The simulations and real-world experiments are both very thorough and nicely done. Details in the supplement were also helpful in contextualizing some of the results. \n4. The paper is very well-written, and the figures are very clean!\n\nWEAKNESSES:\n1. The main question that I found myself struggling with was the order of latent variables across regions. I realize that there in a GP connecting them, but that does not guarantee that the ordering of latents is preserved in anyway across regions, right? Concretely, I am not sure I understand that reading through the columns of $C^m$ can tell us which dimensions are shared and which are not unless there is a post-processing step that reorders the latents (and the columns of $C^m$ respectively) so that the ordering of latents across regions is preserved (example: if latent 1 in regions 2 is correlated with a latent in region 1, then does it occupies the first dimension in the latent vector corresponding to region 1 at any time point?). Perhaps this falls our of the bayesian structure of the model, in which case it would be helpful to clarify this. \n2. It would be nice to also include a literature review / related work section discussing the connections of mDLAG to multi-region LDS based models. Including this in the introduction is also fine, but I do think that it would be helpful to describe the pros and cons of both.\n3. Finally, discussions of the amount of time and data needed to obtain reliable estimates from mDLAG would help practitioners understand the merits of the approach. Currently, there is a figure in the supplement describing the clock time per iteration, but as a reader I do not know how many iterations are needed in practice to fit the model, so having a number describing the overall runtime would be useful.\n\nQUESTIONS:\nIn addition to the questions in the weakness section, I have some other minor questions:\n1. Shouldn't $\\bf{d}^m$ have a prior that requires it to be positive? I'd be curious to hear how the authors deal with this?\n2. I am curious to understand the amount of data needed to reliably estimate the parameters of mDLAG, and so I wonder if the authors have any plots demonstrating accuracy vs # of data points for a given # of regions and population size.\n\nLIMITATIONS:\nI do not envision any societal implications of this work in the short term.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe recent developments in neural recording technologies allow recording from large populations of neurons from multiple brain regions simultaneously. Latent space models are often used to analyze these datasets, but they are generally limited to the study of single or two populations of neurons. This work expands on existing dimensionality reduction methods and introduces a new probabilistic method to characterize interactions across multiple (more than two) populations of neurons. The proposed model can capture local and shared variability across multiple populations as well as the direction of the signal flow between areas and their temporal evolution with trial resolution. The authors validated the method on simulated data and neural data, showing that the model outperforms an existing method without temporal smoothing, GFA, and lesion versions of the model.\n\nSTRENGTHS:\nThe paper is clearly presented and technically sound. The method was tested and shown to work well in simulation and neural data. This method builds on existing latent space models to enable multi-area signal characterization. Understanding inter-area population activity is an interesting problem in neuroscience, and this work proposes a new method that can help explain these emerging datasets. Importantly, even when tested in cases with only two neural populations, for which there are multiple existing methods, the proposed model still outperforms alternatives. This suggests that this tool could be broadly applied to single, two, or more than two areas population recordings. Additionally, when applied to neural data, they demonstrated the power of the approach by reporting new discoveries.\n\nWEAKNESSES:\nWhile the authors show the promise of their method to understand multi-area signals, they overlooked other existing methods that can also capture this information, such as multiset CCA or extensions of Procrustes alignment to multiple datasets. It would be interesting to add a performance comparison to these models. The model assumes linearity and Gaussian observations, but neural activity is often captured in spiking activity, which is better captured by a Poisson model. The authors show no indication of this limitation. Lastly, the computational cost of using a GP prior for temporal smoothing is relatively high compared to alternative methods, which could limit the potential applications. The method seems to be a direct extension of DLAG to multiple areas. If so, an explicit comparison between them could help assess the significance of the work.\n\nQUESTIONS:\nThe simulated data has predetermined parameters, some set to match characteristics of the neural recordings and some others not. For example, the number of neurons in the recordings is in the order of dozens, while in one of the simulations, it is set to 10. What prompted the specific parameter choices? Or more interestingly, how robust is the model to potential changes in the different parameters: number of neurons, noise parameters, temporal delays, and so on? Another additional application listed for future work is the analysis of other signals like behavior. It would be relevant to know in these cases if the model requires similar dynamics or temporal binning for it to work.\n\nLIMITATIONS:\nThe authors address the limitations of linearity and temporal smoothness but fail to address the potential limitations in computational cost and data demands.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends the delayed latents across groups (DLAG) to a recurrent general form (mDLAG), which allows analyzing the contribution of each latent dimension for multiple observational groups. Besides, the newly proposed mDLAG is able to identify the complicated directions of the information flow between groups with the corresponding time delay estimations. Experiments on two synthetic and two empirical datasets show the effectiveness of the mDLAG in analyzing multiple-grouped neural data.\n\nSTRENGTHS:\n* The model derivation is logically clear. The authors provide a very detailed clarification of the notation, parameter set, and generative procedure of this generative model\n* The experiments are exhaustive, including two synthetic and two real-world datasets with very comprehensive analyses.\n* The model is intuitive and in a general form. Its special cases can reduce the model to some famous generative model like the Gaussian process factor analysis (GPFA).\n\nWEAKNESSES:\n* Some math notation is a bit hard to understand, especially the $\\boldsymbol x$. From my understanding, it is a high-dimensional tensor, but the author does not define this tensor in a clear way, e.g., the order of the dimensions.\n\nQUESTIONS:\n* It is hard for me to understand Fig. 1b.\n* Eq. 3, theoretically mean firing rate $\\boldsymbol d^m$ cannot have a Gaussian prior with zero means. A typical way is to model the log or logit of the firing rate since firing rates > 0.\n* Is that $\\boldsymbol x_{n,t}^m$ should be $\\boldsymbol x_{n,t}$, since from Fig. 3b, it seems like you have a shared 7-d latent across all three neural populations A, B, and C?\n* Is the inference very time-consuming?\n* Is there any other models that can solve the same task with some tiny modification, so that the performance can be compared?\n\nLIMITATIONS:\n/", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}