Dynamic Analysis of Higher-Order Coordination in Neuronal Assemblies via De-Sparsified Orthogonal Matching Pursuit

1 Introduction . Coordinated ensemble spiking has been observed in a variety of brain areas , prompting a range of hypotheses about its role in cognitive function . Studies have documented synchronous spiking at all levels of the mammalian visual pathway [ 1 , 2 , 3 ] . Coordinated neural activity has additionally been hypothesized to influence interareal communication and the flow of neural information [ 4 , 5 , 6 , 7 , 8 ] , and been postulated to be mediated by oscillations in local field potentials [ 9 , 10 , 11 ] . The study of synchrony is also closely tied to memory [ 12 , 13 , 14 ] . The prevalence of coordinated spiking and its functional implications for a range of neural processes have motivated both model-free and model-based approaches to its characterization . An intuitive model-free metric is the pairwise correlations of spike trains smoothed by a Gaussian ( or exponential ) kernel [ 15 , 16 ] ; several pairwise distance metrics have been proposed [ 17 ] as alternatives . Though the coherence of pairwise activity can be described , such measures do not capture higher-order coordination , and are limited in the ability to model dynamics in or determine the significance of pairwise coherence without repeated trials . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Statistical models of neuronal ensemble activity transcend the limitation of model-free metrics to pairwise comparisons . Two widely-used approaches are the maximum entropy models and point process generalized linear models ( GLM ) [ 18 , 19 ] . Maximum entropy models describe the state of the neural population only in terms of its instantaneous correlational structure [ 20 , 21 ] . Models are estimated to match observed firing rates and all pairwise ( and potentially higher-order ) correlations simultaneously . Alternatively , point process GLMs for ensemble spiking [ 22 , 23 ] characterize the influence of past population activity , or other relevant covariates . Though useful in estimating functional connectivity [ 24 , 25 ] , each neuron must be assumed conditionally independent due to regularity conditions that prohibit simultaneous spiking events [ 26 , 27 , 28 ] . This can be circumvented by using an equivalent marked point processes ( MkPP ) representation that explicitly models each disjoint simultaneous spiking event , as derived in [ 29 ] and expounded upon in [ 27 ] . MkPP representations of ensemble activity have also been utilized to analyze neuronal population coding in unsorted spiking data [ 30 , 31 ] . A related approach models disjoint simultaneous spiking events as log-linear combinations of point process models , hence permitting an intuitive representation of excess or suppressed synchrony [ 10 , 28 ] . Though statistical models can capture higher-order neural coordination , existing approaches face key limitations . Maximum entropy models can track dynamics in coordination using state-space filtering algorithms , but neglect the influence of past population activity on the ensemble state . Log-linear point process models address this shortcoming , but still share two shortcomings with maximum entropy models . First , assumptions on the relevance of higher-order interactions are typically imposed for tractable model estimation . Second , multiple repeated trials are required to capture dynamics in correlational structure and to evaluate the statistical significance of coordinated spiking . We address these limitations by proposing an adaptive greedy filtering algorithm based on the discretized MkPP formulation in [ 27 ] to model dynamics in coordinated spiking within continuous recordings while capturing the influence of past ensemble activity . Furthermore , we build on recent theoretical results related to Adaptive Granger Causality analysis [ 24 ] to provide a precise statistical framework to detect significantly coordinated activity of arbitrary order . We demonstrate our proposed method ’ s utility in tracking dynamics in coordinated spiking with statistical confidence on simulated ensemble spiking . Applying our method to continuous multi-electrode recordings of human cortical assemblies during anesthesia provides novel insights into coordinated spiking dynamics that underlie transitions between brain states . 2 Preliminaries . In this section , we review the discrete-time marked process representation and two corresponding likelihood models . First , we present a summary of key notation used throughout the subsequent sections in Table 1 . 2.1 Marked Process Representation of Ensemble Spiking . To characterize coordinated spiking , we utilize the discrete-time marked process ( MkPP ) representation of ensemble neuronal activity [ 27 , 28 ] . For an ensemble of C neurons , the C-variate spiking process , binned with small bin size ∆ , at time bin index t is denoted by nt : = [ n ( 1 ) t , n ( 2 ) t , . . . , n ( C ) t ] ′ , where each component is the spiking process of one neuron . Conventional discrete point process models treat the components as conditionally independent Bernoulli observations . Given our interest in simultaneous spikes , we instead treat nt as multivariate Bernoulli observations . The spiking process nt is mapped to a C∗-variate process n∗t : = [ n ∗ t ( 1 ) , n∗t ( 2 ) , . . . , n∗t ( C∗ ) ] ′ , which are the binned observations of a marked point process whose marks count the number of exactly one ofC∗ : = 2C−1 disjoint non-zero spiking events ; we refer to n∗t as the marked Bernoulli process , distinguishing it from the multivariate Bernoulli process nt . We define the mark space K : = { 1 , . . . , C∗ } [ 26 ] . Fig . 1 shows an example of mapping the activity of C = 3 neurons to C∗ = 7 marked processes . At each time tj such that ntj 6= 0 , the sole non-zero element of n∗tj indicates the mark . We also define the binned ground process n ( g ) t that takes value 1 at each such tj and is zero otherwise [ 26 ] ; the ground process indicates the occurrence of any spiking event and is represented by n ( g ) t : = ∑C∗ m=1 n ∗ t ( m ) . The marked process representation is not unique , but can be defined in a convenient fashion : treating the components ofnt as the bits of aC-bit binary number , the mark indexed by the decimal equivalent of a particular realization of nt will corresponded to that realization . By the disjointness of the marked representation , the spiking process of the cth neuron can be recovered as the sum of all marked process whose index , in binary , takes value 1 at the cth bit . For instance , in Fig . 1 , the spiking data of neuron 3 ( in blue ) is the sum of simultaneous spiking event processes 4–7 . Our main contribution in this work is the dynamic and statistically precise inference of latent coordinated spiking of C neurons using their simultaneous spiking representation ( Fig . 1 , bottom panel ) . To this end , we next describe two useful likelihood models for simultaneous spiking . 2.2 Two Likelihood Models of Simultaneous Spiking . In the discrete formulation , the conditional intensity functions ( CIFs ) of nt and n∗t are approximated by the probabilities of observing an event at time bin t given the ensemble ’ s spiking history . That is , λ ( c ) t ∆ = P [ n ( c ) t = 1|Ht ] , λ∗t ( m ) ∆ = P [ n∗t ( m ) = 1|Ht ] , ( 1 ) for c = 1 , . . . , C and m = 1 , . . . , C∗ . We can relate λ ( c ) t ∆ to λ ∗ t ( m ) ∆ in the same manner as n ( c ) t to n∗t ( m ) , and obtain the CIF of the ground process λ∗t ( g ) ∆ = ∑C∗ m=1 λ ∗ t ( m ) ∆ . The marked process permits a generative description of simultaneous spiking events : ensemble spiking events are characterized by the ground process , occurring with probability λ∗t ( g ) ∆ ; the event is then assigned to the mth mark ( i.e . the mth simultaneous spiking outcome ) with conditional probability λ ∗ t ( m ) ∆ λ∗t ( g ) ∆ . Thus , at time t the likelihood of ensemble event n∗t is given by : p ( n∗t ) = C∗∏ m=1 ( λ∗t ( m ) ∆ λ∗t ( g ) ∆ ) n∗t ( m ) ( λ∗t ( g ) ∆ ) n ( g ) t ( 1− λ∗t ( g ) ∆ ) 1−n ( g ) t . ( 2 ) The likelihood in ( 2 ) is used to form a multinomial generalized linear model ( mGLM ) with multinomial logistic link function of which we consider two versions . The first makes the simplifying assumption that there is no history dependence ; the resulting model depends only on contemporaneous spiking , permitting compact parameterization by µt = [ µ ( 1 ) t , µ ( 2 ) t , . . . , µ ( C∗ ) t ] ′ . Defining the baseline firing parameter for the mth mark to be µ ( m ) t : = log ( λ∗t ( m ) ∆ 1− λ∗t ( g ) ∆ ) , m = 1 , 2 , · · · , C∗ , ( 3 ) or equivalently λ∗t ( m ) ∆ = e µ ( m ) t 1+ ∑C∗ j=1 e µ ( j ) t , the log-likelihood can be rewritten as a linear function of n∗t , resembling the maximum entropy model [ 20 , 21 ] : log p ( n∗t ) = µ ′ tn ∗ t − ψ ( µt ) , where ψ ( µt ) : = log ( 1 + C∗∑ m=1 eµ ( m ) t ) . ( 4 ) The second , more general version utilizes the ensemble history as covariates in the mGLM . Letting the covariate vector xt be the ensemble history up to some fixed lag at time t ( augmented by a constant element of 1 ) , the model is parameterized by ωt : = [ ω ( 1 ) ′ t , ω ( 2 ) ′ t , . . . , ω ( C∗ ) ′ t ] ′ , where the parameters for themth mark ω ( m ) t : = [ µ ( m ) t , θ ( m ) ′ t ] ′ consists of an ensemble history-modulation vector θ ( m ) t in addition to the baseline firing parameter . Thus , the log-likelihood in this case admits a similar form to ( 4 ) , by simply replacing µ ( m ) t with x′tω ( m ) t . 3 Adaptive Estimation of the History-Dependent mGLM . Unlike conventional mGLM models , here the parameters are allowed to change in time . To capture their dynamics , we take a similar approach to the dynamic history-independent model of [ 32 ] and extend it to the history-dependent mGLM . We assume conditional independence across time bins and that the parameters ωt admit piece-wise constant dynamics and are constant over consecutive windows of length W . The ensemble history up to lag p defines the covariates as xt : = [ 1 , n ( 1 ) t−1 , . . . , n ( 1 ) t−p , . . . , n ( C ) t−1 , . . . , n ( C ) t−p ] ′ . The set of history covariate vectors at the ith window are denoted byXi = [ x1+i ( W−1 ) , . . . , xiW ] ′ . Since the mapping from n∗t to nt is injective , the influence of past spiking activity can be equivalently captured by defining history covariates in terms of either ; however , using nt reduces the dimensionality of ωt and quantifies the influence of past spiking activity directly rather than through categorical variables . Let n ∗ ( m ) i = [ n ∗ ( m ) 1+W ( i−1 ) , . . . , n ∗ ( m ) iW ] ′ denote the sequence of outcomes of the mth mark in the ith window . The log-likelihood of the ith window is thus given by ` i ( ωi ) : = C∗∑ m=1 n∗i ( m ) ′Xiω ( m ) i − iW∑ j=1+ ( i−1 ) W log ( 1 + C∗∑ m=1 exj ′ω ( m ) i ) . ( 5 ) Motivated by the RLS objective function [ 33 ] , a forgetting factor mechanism is utilized to combine the log-likelihoods up to the kth window , capturing the dynamics in each mark ’ s rates . For a forgetting factor 0 ≤ β < 1 , the adaptively-weighted log-likelihood at window k is thus defined as : ` βk ( ωk ) : = ( 1− β ) k∑ i=1 βk−i ` i ( ωk ) . ( 6 ) Parameter estimation can be performed by solving a sequence of maximum likelihood problems : ω̂k : = arg max ωk ` βk ( ωk ) , k = 1 , 2 , · · · , K. ( 7 ) Two issues arise when considering large ensembles . First , the dimensionality of µk grows exponentially with C ; second , it is likely that some marks will not contain any events . To address this , we employ a thresholding rule similar to [ 34 ] , considering only “ reliable interactions ” , i.e . the subset of the mark space K̄ = { m ∈ K : ∑ t n ∗ t ( m ) > Nthr } for some pre-defined constant Nthr > 0 , and treating the rates of the remaining marked processes as negligible due to their infrequency . To efficiently solve the sequence of problems in ( 7 ) in an online fashion , we develop an adaptive greedy approach based on a generalized Orthogonal Matching Pursuit ( OMP ) [ 35 ] [ 36 ] . The OMP iteratively identifies the parameter support set ( the non-zero components ) of fixed size ( a hyperparameter for which we cross-validate ) to encourage sparsity , thus capturing the inherent sparsity of network interactions based on past ensemble activity [ 37 , 38 , 39 ] . Moreover , the greedy estimation over a sparse subset of parameters mitigates the intractability of the estimation problem for large ensembles where regularization-based constraints still require optimization over all parameters . The proposed adaptive OMP ( AdOMP ) algorithm , so named because the support set is permitted to change between windows , is detailed in Algorithm 1 in Appendix A.1 . The key element of AdOMP is efficient evaluation of the gradient ∇ω ` βk ( ωk ) at the lth iterate ω̂ ( l ) , k , to determine the next addition to the parameter support set and to solve the new maximization problem via gradient descent . Hence , its recursive computation is crucial for the algorithm to operate in an online fashion . To this end , we utilize a recursive update rule to compute the gradient at the kth window , generalizing the adaptive filtering techniques employed in [ 40 ] for Bernoulli observations to a multivariate setting .

Summary: This work develops a set of tools for studying higher order interactions in neural activity recordings. Population neural activity is represented as a marked point process, and fit using a time-varying multinomial GLM. Further, the authors present hypothesis testing tools that compare the fit model with a reduced model that models neurons independently. Specifically, the authors present a Deviance measure that can be used in testing whether r-th order interactions are present in data.