SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Simulation 2: Disentangling concurrent, bidirectional signaling We next considered a simulated scenario that illustrates mDLAG's ability to distinguish concurrent, bidirectional signaling across populations, particularly in contrast with static multi-population methods like group factor analysis (GFA).<sup>2</sup> We again simulated activity from three populations, each with  $q_{m} = 10$  neurons. These populations interacted through two latent processes, one that propagated from population A to B to C (Fig. 4a, Latent 1) and one that propagated in the opposite direction, from population C to B to A (Fig. 4a, Latent 2). Each latent incurred a 15 ms time delay propagating from one population to the next. For the sake of visual demonstration, we generated latent signals with a simple transient peak of activity (Fig. 4a, black traces), and generated the same latent signals on each trial. We then generated observed neural activity according to the mDLAG observation model (equations 1 and 2). Importantly, we scaled the columns of the loading matrices  $C^{m}$  (for each area  $m$ ) so that both columns had equal magnitude, and thus both latent interactions exhibited equal strength in each population: such a communication scheme is difficult to disentangle [15]. Finally, to isolate the consequences of model structure from issues like overfitting, we simulated a low-noise, data-rich scenario by generating  $N = 1,000$  independent trials, each 500 ms in length (comprising  $T = 25$  time points), with high signal-to-noise ratios  $\mathrm{tr}(C^m C^{m\top}) / \mathrm{tr}((\Phi^m)^{-1}) = 10.0$  for each population.
We fit both an mDLAG model and a GFA model (see Supplementary Section S3) to this simulated neural activity, assuming for each model the correct number of latent variables,  $p = 2$ . mDLAG latent variable and time delay estimates accurately reflected the distinct signaling pathways across
See also Supplementary Fig. S2b for performance comparisons between GFA and mDLAG on the neural recordings analyzed in Section 4.
Figure 4: mDLAG disentangles concurrent signaling where static methods do not. (a) Schematic of simulated ground truth. Populations A, B, and C interact through two latents. Latent 1 (left) propagates from A to B to C. Latent 2 (right) propagates from C to B to A. (b) mDLAG estimates disentangle the two interactions. Each panel corresponds to a single latent variable. Each black trace represents one trial (10 representative trials shown). For concision, we show only latents corresponding to one population  $(\mathbf{x}_{n,j,:}^{m})$ ; the remaining latents are time-shifted versions of those shown here. Inset above each latent are the involved populations along with the signal flow and magnitude of time delays between populations (in ms). a.u.: arbitrary units. (c) Group factor analysis (GFA) estimates represent mixtures of the two interactions. Same conventions as in (b).
the three populations (Fig. 4b;  $R^2$  between ground truth and estimated time courses: 0.989; mean delay error: 0.16 ms). Each latent estimated by GFA, however, notably reflected a mixture of both interactions (Fig. 4c, each latent time course exhibits two peaks). With no description of signal flow, static methods like GFA struggle to distinguish concurrent, bidirectional signaling.
Validating mDLAG on recordings from V1 and V2 To further validate mDLAG, we turned to neural recordings from two areas in the macaque visual cortex, V1 and V2 [24]. These recordings have been studied extensively [15, 25-27], and therefore provide an excellent testbed for our framework. We applied mDLAG to the simultaneously recorded activity of neuronal populations in the superficial (output) layers of V1 (61 to 122 neurons; mean 86.3), and the middle (input) layers of V2 (15 to 32 neurons; mean 19.6) in three anesthetized animals (Supplementary Fig. S1a). We analyzed neuronal responses measured during the 1.28 second presentation of drifting sinusoidal gratings of different orientations, and counted spikes in  $20~\mathrm{ms}$  time bins. Because we were interested in V1-V2 interactions on timescales within a trial, we subtracted the mean across time bins within each trial from each neuron. This step removed activity that fluctuated on slow timescales from one stimulus presentation to the next [28]. In total, we analyzed separately 40 "datasets," corresponding to five recording sessions, each with eight different orientations. Each dataset included 400 trials: we allocated at random 300 trials as a training set and 100 trials as a test set on which to measure model performance (we employed a leave-group-out prediction metric; see Supplementary Section S4).
Given prior work on these V1-V2 recordings, we reasoned that mDLAG ought to qualitatively recover several hallmarks of the V1-V2 activity: (1) a significant number of latent variables local to each area, indicative of selective V1-V2 communication (Supplementary Fig. S1b, 'V1' and 'V2'; Supplementary Fig. S1c, latents 4 and 5), (2) the presence of periodic structure representative of the drifting grating stimulus (Supplementary Fig. S1c, latents 3 and 4), and (3) bidirectional interactions (Supplementary Fig. S1c, latents 1-3). Indeed, mDLAG detected all of these hallmarks.
These recordings were also previously studied with DLAG [15], which was designed to study concurrent signaling between two neuronal populations. Since DLAG is a special case of the mDLAG framework, we reasoned that mDLAG ought to exhibit similar quantitative performance and extract qualitatively similar features of V1-V2 communication. Indeed, the latent time courses estimated by mDLAG (Supplementary Fig. S1c) were qualitatively similar to those estimated by DLAG [29] (Supplementary Fig. S1d). Interestingly, mDLAG outperformed DLAG across all datasets (Supplementary Fig. S2a, points above the diagonal), suggesting that ARD provides an improved method of model selection over the constrained grid search method used for DLAG, while also avoiding grid search's computational drawbacks (see Supplementary Section S5). Collectively, the results across our simulations and the V1-V2 recordings indicate that mDLAG is well-equipped to study concurrent signaling across multiple simultaneously recorded neuronal populations.
Figure 5: Applying mDLAG to Neuropixels recordings across V1, V2, and V3d. (a) Schematic showing a sagittal section of occipital cortex and the recording setup. Laminar compartments of V1, V2, and V3d were recorded using multiple Neuropixels probes. (b) Temporally smoothed peristimulus time histograms during the stimulus presentation period, for an example session and stimulus condition. (c) Schematic showing the subset of laminar compartments included in the analyses (shaded compartments). Compartments with the same color were treated as one population during analysis. Arrows indicate prominent anatomical connections [30]. (d) Visual receptive field locations of V1, V2, and V3d. Each circle represents receptive field size and location across the channels of a single Neuropixels probe. Scale bars indicate degrees of visual field. (e) Total number of latent dimensions shared between each pair of populations. Light gray: dimensions unique to a pair. Dark gray: dimensions shared globally across all populations. (f) Single-trial time course estimates of select shared latents (same dataset as in (b), (d), and (e)). Same conventions as in Fig. 4b.
4 Interactions across laminar compartments of V1, V2, and V3d
We next used mDLAG to study interactions across V1, V2, and V3d of anesthetized macaques recorded using multiple Neuropixels probes (Fig. 5a). These recordings included hundreds of neurons across different layers in each area (Fig. 5b, superficial (S), middle (M), and deep (D) compartments) allowing us to investigate longstanding questions about functional cortical organization.
First, we sought to quantify the performance of mDLAG models relative to alternative approaches, and to further demonstrate the empirical benefit of mDLAG's model components. We therefore considered three types of models: group factor analysis (GFA); mDLAG, but with time delays removed ('mDLAG-0'); and mDLAG. With each model type, we analyzed spike counts (in 20 ms bins) of neurons in select laminar compartments of each area, measured during the first 1,000 ms after the onset of a drifting sinusoidal grating (Fig. 5b, trial-averaged responses for an example dataset). The selected laminar compartments (grouped into three sub-populations for analysis: Fig. 5c, color shading) were those that likely directly interacted through a combination of feedforward, feedback, and inter-laminar connections [30] (Fig. 5c, black arrows). As we did for the V1-V2 recordings, above, we subtracted the mean across time bins within each trial from each neuron, to remove slow fluctuations beyond the timescale of a trial. We fit GFA, mDLAG-0, and mDLAG separately to 10 datasets, comprising five recording sessions (each with two grating orientations,  $90^{\circ}$  apart, presented 300 trials each) from two anesthetized animals. For each dataset, we allocated at random 225 trials as a training set and 75 trials as a test set on which to measure model performance. GFA, mDLAG-0, and mDLAG exhibited increasingly better leave-group-out prediction (Supplementary Fig. S2b: mDLAG-0 better than GFA; c: mDLAG better than mDLAG-0), demonstrating the performance benefit of (1) temporal smoothing and (2) time delays (see also Supplementary Fig. S3 for a demonstration of mDLAG performance versus number of available training trials, and Supplementary Fig. S4 for a characterization of GFA and mDLAG runtimes).
We then focused on one particular set of questions: Do V1, V2, and V3d communicate selectively, and if so, what aspects of the recorded populations might contribute to this selectivity? One likely aspect is the alignment of visual receptive fields (RFs): retinotopically aligned neuronal populations might co-contribute to computations that non-aligned populations would not. This functional organization would appear as mDLAG latent variables shared exclusively between the aligned populations. We therefore studied a dataset from a recording session in which the retinotopy of the recorded populations allowed us to investigate this hypothesis (Fig. 5d). RFs of the V1 and V3d populations were largely overlapping (Fig. 5d, blue: V1, gold: V3d), whereas the RFs in V2 did not overlap with either V1 or V3d (Fig. 5d, red: V2).
We quantified the total number of dimensions shared between each pair of populations (Fig. 5e; a dimension was considered significant in a population if it explained at least  $2\%$  of the shared variance, Supplementary Section S5, equation S52). mDLAG allowed us to distinguish dimensions that were unique to a pair of populations (e.g., between V1 and V3d, but not V2) from dimensions that also involved the third population. V1-V3d interactions included unique dimensions (Fig. 5e, 'V1-V3d', light gray) in addition to a dimension shared globally across all three populations (Fig. 5e, 'V1-V3d', dark gray), whereas V1-V2 interactions and V2-V3d interactions could be attributed primarily to the global dimension (Fig. 5e, 'V1-V2' and 'V2-V3d', only dark gray). The single-trial time courses of the global latent evolved according to a long timescale (Fig. 5f, latent 1,  $132~\mathrm{ms}$ ) relative to the unique V1-V3d latents (Fig. 5f, latents 2 and 3, 8 ms and  $62~\mathrm{ms}$ , respectively). The global latent was also associated with a direction of signal flow (V3d to V2 to V1) opposite to that of the V1-V3d latents (V1 to V3d). The time courses of the V1-V3d latents exhibited features of the stimulus response, including a fast transient response (Fig. 5f, latent 2) and periodic structure with the same period as the drifting grating stimulus (Fig. 5f, latent 3,  $333~\mathrm{ms}$  period). Results were largely insensitive to the initialization of the mDLAG fitting procedure (Supplementary Fig. S5). These findings are consistent with the hypothesis that visual cortical populations communicate selectively in a retinotopic manner, as they perform computations on inputs from localized regions of visual space [31].
Even for two populations, identifying which latents are shared between populations or local to a population is a challenging computational problem. Existing approaches explicitly designate latents as shared or local [13-15], and then rely on heuristics to avoid the computational cost of grid search. Scaling this type of approach to three or more populations (or external experimental variables) would be prohibitive. mDLAG is thus an advance toward scaling to large-scale multi-population recordings, and could continue to be improved through the many approaches used to scale Gaussian process methods in neuroscience [13, 23, 32-36].
mDLAG treats time delays as constant parameters across trials, time, and neurons. The delay estimated for each latent variable thus represents a summary of inter-population interaction throughout the course of an experiment. To assess the variability of delay estimates, one could fit mDLAG to subsets of trials, subsets of neurons, or to separate trial epochs. If additional subpopulation labels are available (for example, laminar compartments in our Neuropixels recordings, Fig. 5b), one could also specify these subpopulations to the mDLAG model to better incorporate variability of delays across neurons.
A latent variable shared across three or more populations is potentially consistent with multiple signaling schemes. For instance, Latent 1 of Fig. 5f is consistent with a signal relayed from V3d to V2 (with a delay of  $2\mathrm{ms}$ ) and then from V2 to V1 (with a delay of  $4\mathrm{ms}$ ). It is also consistent, however, with a scheme in which V3d is a common input to V2 (with a delay of  $2\mathrm{ms}$ ) and to V1 (with a delay of  $6\mathrm{ms}$ ). A third scheme could involve common input to all three areas from an unobserved source. Still, mDLAG narrows the set of populations that could be involved in any given interaction, and the sign and magnitude of mDLAG's time delays narrow the set of signaling schemes consistent with the data. This hypothesis set can be narrowed further by experimental interventions [37].
The mDLAG model includes assumptions of linearity and temporal smoothness; specifically, we have employed here the commonly used squared exponential function for mDLAG's GP covariances. It might be desirable to incorporate temporal structure more appropriate for certain signals (for example, the sinusoidal structure of Latent 3, Fig. 5f). Then, an alternative GP covariance function can be substituted in a straightforward manner into the mDLAG model (equation 8) [19, 32, 33, 38].
Nonlinearities, for example related to neuronal tuning or receptive fields, could also be incorporated under a GP framework [39, 40].
For exploratory data analysis, mDLAG's GP-based description of multi-population temporal structure is advantageous over an alternative linear dynamical system (LDS)-based description [10, 11] in two respects: (1) a GP can be useful for exploratory data analyses where an appropriate parametric dynamical model is unknown a priori, and (2) mDLAG's continuous-time model enables the discovery of wide-ranging delays with high precision, which, in contrast to discrete-time LDS approaches, are not limited to be integer multiples of the sampling period or spike count bin width of the neural activity. Ultimately, these approaches can be complementary: one can use mDLAG to generate data-driven hypotheses about motifs of concurrent signaling across populations, and then test these hypotheses with a dynamical system-based approach.
Finally, while we applied mDLAG to the spiking activity of neuronal populations, mDLAG is applicable to any high-dimensional time series data, including other neural recording modalities. In fact, the groups analyzed by mDLAG need not all be neuronal populations, but could include, for example, a collection of dynamic stimulus or behavioral variables. mDLAG is a general framework that advances the study of concurrent signaling throughout the brain.
Code availability
A MATLAB (MathWorks) implementation of mDLAG is available on GitHub at http://github.com/egokcen/mDLAG and on Zenodo at https://doi.org/10.5281/zenodo.10048163 [41].
We thank A. Zandvakili for providing the V1-V2 recordings in Supplementary Fig. S1. This work was supported by the Dowd Fellowship (E.G.), ECE Benjamin Garver Lamme/Westinghouse Fellowship (E.G.), Simons Collaboration on the Global Brain 542999 (A.K.), 543009 (C.K.M.), 543065 (B.M.Y.), NIH RF1 NS127107 (A.K., C.K.M., B.M.Y.), NIH R01 EY035896 (B.M.Y., A.K., C.K.M.), NIH U01 NS094288 (C.K.M.), NIH CRCNS R01 MH118929 (B.M.Y.), NSF NCS DRL 2124066 (B.M.Y.), NIH R01 NS129584 (B.M.Y.).
[1] Ahrens, M. B. et al. Brain-wide neuronal dynamics during motor adaptation in zebrafish. Nature 485, 471-477 (2012).
[2] Yang, W. & Yuste, R. In vivo imaging of neural activity. Nature Methods 14, 349-359 (2017).
[3] Steinmetz, N. A. et al. Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings. Science 372, eabf4588 (2021).
[4] Semento, J. D., Gokcen, E., Machens, C. K., Kohn, A. & Yu, B. M. Statistical methods for dissecting interactions between brain areas. Current Opinion in Neurobiology 65, 59–69 (2020).
[5] Kang, B. & Druckmann, S. Approaches to inferring multi-regional interactions from simultaneous population recordings. Current Opinion in Neurobiology 65, 108-119 (2020).
[6] Keeley, S. L., Zoltowski, D. M., Aoi, M. C. & Pillow, J. W. Modeling statistical dependencies in multi-region spike train data. Current Opinion in Neurobiology 65, 194–202 (2020).
[7] Machado, T. A., Kauvar, I. V. & Deisseroth, K. Multiregion neuronal activity: the forest and the trees. Nature Reviews Neuroscience 23, 683-704 (2022).
[8] Zhuang, X., Yang, Z. & Cordes, D. A technical review of canonical correlation analysis for neuroscience applications. Human Brain Mapping 41, 3807-3833 (2020).
[9] Bach, F. R. & Jordan, M. I. A Probabilistic Interpretation of Canonical Correlation Analysis. Technical Report No. 688, Department of Statistics, University of California, Berkeley (2005).
[10] Semedo, J., Zandvakili, A., Kohn, A., Machens, C. K. & Yu, B. M. Extracting Latent Structure From Multiple Interacting Neural Populations. Advances in Neural Information Processing Systems 27, 2942-2950 (2014).
[36] Dowling, M., Zhao, Y. & Park, I. M. Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains. Proceedings of the 40th International Conference on Machine Learning (2023).
[37] Reid, A. T. et al. Advancing functional connectivity research from association to causation. Nature Neuroscience 22, 1751-1760 (2019).
[38] Wilson, A. & Adams, R. Gaussian Process Kernels for Pattern Discovery and Extrapolation. Proceedings of the 30th International Conference on Machine Learning, 1067-1075 (2013).
[39] Wu, A., Roy, N. A., Keeley, S. & Pillow, J. W. Gaussian process based nonlinear latent structure discovery in multivariate spike train data. Advances in Neural Information Processing Systems 30, 3496-3505 (2017).
[40] Duncker, L., Ruda, K. M., Field, G. D. & Pillow, J. W. Scalable Variational Inference for Low-Rank Spatiotemporal Receptive Fields. Neural Computation 35, 995–1027 (2023).
[41] Gokcen, E. mDLAG (v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.10048163 (2023).