Title: Bayesian nonparametric (non-)renewal processes for analyzing neural spike train variability
Venue: NeurIPS 2023
Accepted: yes
Keywords: Gaussian processes

renewal processes

point processes

neural data analysis

Bayesian machine learning

non-stationary time series
Abstract: Neural spiking activity is generally variable, non-stationary, and exhibits complex dependencies on covariates, such as sensory input or behavior. These dependencies have been proposed to be signatures of specific computations, and so characterizing them with quantitative rigor is critical for understanding neural computations. Approaches based on point processes provide a principled statistical framework for modeling neural spiking activity. However, currently, they only allow the instantaneous mean, but not the instantaneous variability, of responses to depend on covariates. To resolve this limitation, we propose a scalable Bayesian approach generalizing modulated renewal processes using sparse variational Gaussian processes. We leverage pathwise conditioning for computing nonparametric priors over conditional interspike interval distributions and rely on automatic relevance determination to detect lagging interspike interval dependencies beyond renewal order.  After systematically validating our method on synthetic data, we apply it to two foundational datasets of animal navigation: head direction cells in freely moving mice and hippocampal place cells in rats running along a linear track. Our model exhibits competitive or better predictive power compared to state-of-the-art baselines, and outperforms them in terms of capturing interspike interval statistics. These results confirm the importance of modeling covariate-dependent spiking variability, and further analyses of our fitted models reveal rich patterns of variability modulation beyond the temporal resolution of flexible count-based approaches.

Bayesian nonparametric (non-)renewal processes for analyzing neural spike train variability

Abstract

Neural spiking activity is generally variable, non-stationary, and exhibits complex dependencies on covariates, such as sensory input or behavior. These dependencies have been proposed to be signatures of specific computations, and so characterizing them with quantitative rigor is critical for understanding neural computations. Approaches based on point processes provide a principled statistical framework for modeling neural spiking activity. However, currently, they only allow the instantaneous mean, but not the instantaneous variability, of responses to depend on covariates. To resolve this limitation, we propose a scalable Bayesian approach generalizing modulated renewal processes using sparse variational Gaussian processes. We leverage pathwise conditioning for computing nonparametric priors over conditional interspike interval distributions and rely on automatic relevance determination to detect lagging interspike interval dependencies beyond renewal order. After systematically validating our method on synthetic data, we apply it to two foundational datasets of animal navigation: head direction cells in freely moving mice and hippocampal place cells in rats running along a linear track. Our model exhibits competitive or better predictive power compared to state-of-the-art baselines, and outperforms them in terms of capturing interspike interval statistics. These results confirm the importance of modeling covariate-dependent spiking variability, and further analyses of our fitted models reveal rich patterns of variability modulation beyond the temporal resolution of flexible count-based approaches.

1 Introduction

Analyses of spike time [49, 76, 86] and count [12, 26, 57] statistics have revealed neural responses in vivo to be structured but generally variable or noisy [24, 44, 70, 78]. To model this stochastic aspect of neural spike trains, probabilistic approaches based on temporal point processes have been widely applied. This in turn has been a major driver of point process theory development [42] for capturing spiking variability structure with statistical models [74]. The study of neural computation underlying naturalistic behavior in particular involves non-stationary spike trains, which presents a significant challenge as apparent spiking variability is a result of both irreducible 'intrinsic' neural stochasticity as well as dependencies on behavioral covariates that can themselves vary on multiple time scales.

Different approaches have been proposed for handling non-stationary spike trains, starting with the classical log Cox Gaussian process [14, 55] to allow variations in the local intensity or firing rate while modeling independent spikes. Dependencies on previous spikes can be captured to first order with renewal processes, and these models have been extended to non-stationary cases through modulation of the hazard function with some time-dependent function [43, 79] or through rescaling interspike intervals with a covariate-dependent rate function [2, 6]. Another approach based on Hawkes processes and spike-history filters [45, 83, 91] introduces conditional point processes that go

beyond the first order Markov assumption. Approaches based on recurrent networks [51, 90] and neural ODEs [10, 41] can in theory capture arbitrarily long dependencies on past spikes and input covariates, but provide more limited descriptive interpretability.

However, not only the rate but also the variability of spiking encodes task-relevant information [34, 60], and bears signatures of the underlying computations [11]. Importantly, this variability has stimulus- and state-dependent structure [12, 21, 48, 67]. In statistical modeling language, this corresponds to heteroscedastic or input-dependent observation noise. Such structure reflects computations performed in the underlying neural circuit, and thus characterizing it from data in a flexible and robust manner is critical for advancing theories of neural computation. The classical approaches reviewed above do not attempt to characterize such covariate-dependent changes in variability. Flexible count models have been introduced to more faithfully capture variability at the count level [28], and recent work has extended this to the general case of input-dependent variability [48]. Count approaches however are limited in the resolution of the analysis set by the time bin size. In addition, the resulting count statistics strongly depend on the chosen time bin size [48, 73].

While firing rates are routinely modeled as input-dependent, extending point process models with input-dependent variability has not been widely explored in the literature. Rate-rescaled and modulated renewal processes rely on fixed base renewal densities. Allowing the shape parameters of the renewal density to vary with covariates in the corresponding hazard function is one potential approach, but this still relies on a commitment to a particular parametric family of renewal densities. Spike-history filters in conditional point processes are conventionally fixed and thus do not directly model input-dependent spiking variability, though dependence on observed and unobserved covariates [91] and switching filters based on discrete states [23] have been considered. Recent work has moved away from parametric filters to nonparametric Gaussian processes [18], which can be extended to flexibly model dynamic filters as functions of external covariates with a spatio-temporal Gaussian process. However, any modulation of the filter will no longer permit fast convolutions, and such models will be computationally expensive as the filter needs to be recomputed every time step. The direct nonparametric estimation of conditional intensity functions based on maximum likelihood has been explored [13, 82], but scalable Bayesian approaches have remained absent.

Contribution To enable flexible modeling as well as modulation of instantaneous point process statistics for analyzing neural spike train variability, we introduce the Bayesian nonparametric nonrenewal process (NPNR). NPNR builds on sparse variational Gaussian processes and defines a nonparametric prior over conditional interspike interval distributions, generalizing modulated renewal processes with nonparametric renewal densities and spike-history dependencies beyond renewal order. In particular, our point process can flexibly model modulations of not only spiking intensity but also variability. We validate our model using parametric inhomogeneous renewal processes, recovering conditional interspike interval distributions and identifying renewal order in spike-history dependence. On neural data from mouse thalamus and rat hippocampus, our method has competitive predictive power while being superior in capturing interspike interval statistics from the non-stationary data. In particular, our method provides instantaneous measures of spike train variability that are modulated by covariates, and shows rich variability patterns in both datasets consistent with previous studies at coarser timescales. We provide a JAX [4] implementation of our method as well as established baseline models within a scalable general variational inference scheme. 1

2 Background

We start with a brief overview of the theoretical foundations and related point process models, as well as their combination with Gaussian processes to introduce non-stationarity.

2.1 Temporal point processes

Statistical modeling of events that occur stochastically in time is handled by the general framework of temporal point processes [59, 72]. Denoting the number of events that occurred until time t by N ( t ) , a temporal point process model is completely characterized by its conditional intensity function (CIF)

where λ ( t ) δt is the probability to emit a spike event in the infinitesimal interval [ t, t + δt ) conditioned on H t , the spiking history before t . We can write the point process likelihood for a single neuron spike train consisting of an ordered sequence of S spike events at times t i as [5]

In neuroscience applications, one often wants to describe modulation of the point process statistics with some time-varying covariates x ( t ) , such as animal head direction or body position, which leads to a generalized CIF λ ( t |H t , x ≤ t ) . Several classes of models have been proposed that are defined by particular restrictions on the functional form of λ ( t |H t , x ≤ t ) .

2.1.1 Inhomogeneous renewal processes

Renewal assumption The statistical model in Eq. 2 describes dependencies between all spikes. One common simplification is the renewal assumption: interspike intervals (ISIs) ∆ ( i ) = t i +1 -t i are drawn i.i.d. from an interval distribution called the renewal density g (∆; θ ) with parameters θ . This induces a Markov structure p ( t i | t i -1 , t i -2 , . . . ) = p ( t i | t i -1 ) in the spike train likelihood

Common renewal densities used for neural data are the exponential (equivalent to a Poisson process), gamma, and inverse Gaussian distributions [2, 6].

Hazard function modulation Non-stationary point processes need to model changes in statistics with time, and combined with the renewal assumption one obtains inhomogeneous renewal processes. A classical approach that dates back to Cox [14] is to modulate the hazard function (Appendix A)

with time since last spike τ = t -t i and the modulation factor ρ ( t ) . In our context, this can be replaced by some function of covariates ρ ( x t ) [43]. A multiplicative interaction between ρ and h as above is typically considered, though this framework allows general parametric forms [71, 79].

Rate-rescaling Another approach that has been widely applied in the neuroscience community is rate-rescaling [2, 6], closely related to time-rescaling (see Appendix B.3). Here, modulation is achieved with a rate function r ( x t ) ≥ 0 that transforms time t into rescaled time ˜ t

This maps all spike times t i to rescaled times ˜ t i , and will be one-to-one as long as r ( t ) > 0 . By modeling the rescaled ISIs ˜ ∆ ( i ) = ˜ t i +1 -˜ t i as drawn from a stationary renewal density g ( · ) , we obtain an inhomogeneous renewal process from a homogeneous one. The CIF becomes dependent on the covariate path since last spike P k = { x ( u ) | u ∈ ( t i , t ]) } , see Appendix B.3.

2.1.2 Conditional point processes

Conditional Poisson processes The renewal assumption ignores correlations between ISIs, which generally are observed in both the peripheral and central nervous system [1, 25] and can be computationally relevant for signal detection and encoding [8, 9, 22, 69]. Going beyond Markovian dependencies, a tractable approach, similar to Hawkes processes [51], is to introduce a causal linear filter h ( t ) that is convolved with spikes and added to the log CIF, giving conditional Poisson processes

where ∗ denotes temporal convolution. These models are closely linked to mechanistic integrate-andfire models [52, 85] and have a long history in the neuroscience literature [18, 31, 35, 47, 62, 66, 83], appearing as generalized linear models (GLMs) and spike response models (SRMs).

Conditional renewal processes An even more expressive model can be obtained by replacing the Poisson spiking process with a rate-rescaled renewal process. This results in a conditional renewal process, where the rate function has history dependence r ( t |H t , x t ) as in Eq. 6.

2.2 Gaussian process modulated point processes

Gaussian processes (GPs) represent a data-efficient alternative to neural networks, which have been widely used to model the CIF [59, 72, 90]. When combining GPs with point process likelihoods, the resulting generative model leads to doubly stochastic processes for event data. Placing a Gaussian process prior [87] over the log intensity function leads to the classic log Cox Gaussian processes [14, 55], and in the same spirit one can modulate renewal hazard functions [79] or perform raterescaling [16] with GPs. Such constructions form the basis of many widely used Bayesian neural encoding models for spike trains, both for modeling single neuron responses [15, 16, 68] as well as population activity [19, 93]. Combining the flexibility offered by renewal and conditional point processes with GP rate or modulation functions within a variational framework has been impeded by the fact that the original papers were built on a GLM framework with parametric covariate mappings [6, 7, 18, 65]. To provide a fair comparison of NPNR to these baselines, we implement a general scalable variational inference framework for the construction and application of such models (see Appendix B for details on the baseline models).

3 Method

We now introduce the nonparametric non-renewal (NPNR) process and present the approximate Bayesian inference scheme used for model fitting, noting connections to related works in the literature. Our NPNR model provides a nonparametric generalization of modulated renewal processes beyond renewal order, and adds suitable inductive biases for neural spike train data. It implicitly defines a flexible prior over conditional ISI distributions that can be computed using pathwise conditioning, which enables one to analyze spiking variability modulation with minimal parametric constraints. Furthermore, the Bayesian framework provides an elegant data-driven approach to inferring the lagging ISI order of the spike-history dependence.

3.1 Generative model

Conditional intensity surface priors To obtain flexible modulated point process models, we directly model the CIF, or more precisely its logarithm, of the form

where t i is the most recent spike at current time t . First considering the renewal case, we note the spatio-temporal structure in the log CIF using time since last spike τ = t -t i is

which suggests placing a spatio-temporal GP prior on the log CIF to describe a log intensity surface

This generalizes the parametric forms of modulation considered in previous approaches [43, 79], in particular allowing modulation of the effective instantaneous renewal density by covariates x t . We can introduce lagging ISIs covariates ∆ k ( t ) with lag k as depicted in Fig. 1A to extend the model to a non-renewal process

and for a maximum ISI lag K we denote the lagging ISIs ∆ t = [∆ 1 ( t ) , . . . , ∆ K ( t )] to obtain

Inductive biases for neural data For neural spiking data, there are biological properties to consider for building a more realistic prior. Firstly, neurons have refractory periods immediately following a spike, though in practice neural recordings may not respect this due to contamination in spike sorting [38]. Another potentially useful inductive bias is that changes in the spiking intensity of

neurons fluctuate mostly at shorter ISI timescales [32], whereas at longer delays they tend to be temporally smoother. The latter suggests non-stationary GP kernels to be more suitable for modeling the spike-history dependencies [18]. However, non-stationary kernels do not allow straight-forward use of random Fourier features for evaluating GP posterior function samples at many locations with pathwise conditioning [88, 89]. This in particular is needed to compute the conditional ISI distributions g ( τ | . . . ) in Eq. 15, see Appendix B.5 for details. To achieve the desired non-stationarity for modeling Eq. 11 while maintaining the ability to draw samples using pathwise conditioning, we apply time warping on τ from [0 , ∞ ) → [0 , 1] with some warping timescale τ w

and place a stationary Gaussian process prior over the warped temporal dimension f (˜ τ, . . . ) ∼ GP with a temporal kernel k (˜ τ, ˜ τ ′ ) = k (˜ τ -˜ τ ′ ) . This transformation is monotonic (see Fig. 1B), and hence we can easily compute the transformation on the CIF

Similarly, we apply time warping to the ∆ k dimensions on which we also place stationary kernels k ( ˜ ∆ , ˜ ∆ ′ ) = k ( ˜ ∆ -˜ ∆ ′ ) . We note that unlike spike-history filters in conditional point processes (Eq. 6) which do not change with inputs, the resulting coupling to past activity in Eq. 11 is dependent on covariates x t . This allows one to capture spiking variability modulation via the conditional ISI distribution perspective discussed below. The refractory nature of real neurons can be addressed by the mean function

with parameters a m , τ m , b m , where refractory periods can be modeled with large negative a m .

Conditional ISI distributions Instead of looking at the CIF, we can view the model as a prior over conditional ISI distributions as depicted in Fig. 1C using the relation (see Appendix A)

where one drops the dependence on H t in g for the modulated renewal case. If one fixes the lagging ISIs and picks a constant covariate path g ( τ | ∆ ∗ , x ∗ ) , this can be interpreted as an instantaneous ISI distribution of a neuron at the conditioned inputs ∆ ∗ and x ∗ . Moments of the conditional ISI distribution are computed using Gauss-Legendre quadratures in warped time (Appendix B.5)

and this can be used to compute tuning curves of spike train statistics, such as the mean ISI E [ τ ] and coefficient of variation CV = √ Var [ τ ] / E [ τ ] [58], as a function of x ∗ . This approach generalizes the homogeneous case considered in the literature, and in particular allows one to compute instantaneous measures of non-stationary spike train variability that are otherwise non-trivial to estimate [58, 75].

3.2 Inference

Temporal discretization The generative model is formulated as a continuous time model. In practice, neural and behavioural data are typically recorded with finite temporal resolution at small regular intervals ∆ t . The cumulative intensity integral has to be approximated by a sum, though note that directly modeling the cumulative hazard function [59] elegantly avoids this for purely temporal point processes. Spike times are now discretized as a binary vector y = [ y 1 , . . . , y T ] where y t = 1 if t has a spike event, zero otherwise. Overall, this discretizes the point process likelihood Eq. 2 as

where we have T time steps in total. Note that the discretization scheme implies we do not have observations at τ = 0 , since the time step immediately after an observed spike has τ = ∆ t .

̃

τ

Δ1

Δ2

x

1

time

x

2

time

0.0

0.3

τ

(s)

density

time

0

1

τ

(a.u.)

density

τ

0

1

τ

τw

=⟨

ISI

⟩

0

1

τ

(a.u.)

-3

0

log CIF

non-refractory prior

(

am

=0

)

0

1

-5

0

refractory prior

(

am

= -6

)

0

3

τ

(s)

ISI distribution

0

3

A

B

C

Variational lower bound We use stochastic variational inference [39] with batches obtained from consecutive temporal segments and sparse variational GPs [37], giving the loss objective

where n indexes neurons (of which there are N ) 2 , u = [ u 1 , . . . , u M ] denotes the set of M inducing points, p ( u ) the GP prior at inducing locations, q ( u ) the variational posterior, and q ( f | u ) the conditional posterior (see Appendix B.1 for details). Combined with temporal mini-batching to fit batch segments of length T , we can fit to very long time series given the O ( N T M 2 + N M 3 ) computational complexity. We also no longer rely on computing hazard functions of parametric renewal densities to obtain the CIF, which can be numerically unstable. Modulated renewal processes instead rely on a specialized thinning procedure [79], but we take a more scalable and general variational approach. Overall, we optimize the kernel hyperparameters, variational posterior mean and covariance, inducing point locations, and mean function parameters a m , b m and τ m using gradient descent with Adam [46] (see Appendix C for details). The time warping parameter τ w is fixed in our experiments to the empirical mean ISI, and the hyperparameter K is fixed and chosen in advance (see also subsection on automatic relevance determination below).

Automatic relevance determination The Bayesian framework with GPs enables us to perform automatic relevance determination (ARD) over the input dimensions to automatically select relevant input [37, 80]. Applied to lagging ISI dimensions in our NPNR model, this provides an elegant approach to making a data-driven renewal assumption and generally determining the spike-history dependence of the CIF. We choose to fix τ w to the empirical mean ISI as shown in Fig. 1B (rather than learning it) to achieve interpretability of kernel timescales for ARD (Fig. 6A) at a small cost of performance (Fig. 12). For a chosen maximum lag K , there is no need for manual selection of the history interaction window as for GLM spike-history filters, though recent work on nonparametric GLMfilters provides a related window size selection procedure [18]. In the spirit of Bayesian models, we choose K to give a sufficiently high capacity model [40, 80] to be able to flexibly capture history dependence, as seen in panel D of Fig. 3 and Fig. 4.

4 Results

All datasets discretize spike trains and input time series at regular intervals of ∆ t = 1 ms. We use a product kernel for k ( x , x ′ ) with periodic kernels for angular dimensions, and squared exponential kernels in other cases. For k (˜ τ, ˜ τ ′ ) and k ( ˜ ∆ , ˜ ∆ ′ ) , we pick a product kernel with Matérn3 / 2 (see Fig. 12 for different kernel choices) and set the maximum ISI lag K = 3 . For illustration, conditional

̃

̃

S

density

true renewal shapes

ISI (a.u.)

0

1

ISI (s)

probability

1

5

9

neuron index

10

0

10

1

10

2

lag 0

lag 1

lag 2

lag 3

1234

order

frequency

0

1

Poisson

0

1

0

1

rescaled gamma

conditional Poisson

0

1

NPNR (ours)

A

true rate maps

B

inferred rates

C

quantile

empirical distribution function

S

ISI distributions and corresponding tuning curves are computed by fixing ∆ k to be the mean ISI per neuron. Firing rates are defined as 1 / E [ τ ] , since this corresponds to the number of spikes fired per unit time in infinitely large time bins for a renewal process (Eq. 26). GP inducing points were randomly initialized, and for a fair comparison, all models used 8 inducing points for each covariate dimension (including temporal dimensions τ and ∆ in the NPNR process). For each experiment, we repeat model fitting with 3 different random seeds and pick the model with the best training likelihood. Further details on experiments are presented in Appendix C.

4.1 Validation on synthetic data

For validating our approach, we generate 1000 s of data using rate-rescaling [65] mimicking a place cell population of 9 neurons for an animal moving in a 2D square arena, each with a unique rate map and renewal density (Fig. 2A, details in Appendix C). The models applied are baseline Poisson, raised cosine filter conditional Poisson and rate-rescaled gamma processes (Appendix C), and our NPNR process. Note that the rescaled gamma process is within-model class for 3 of the synthetic neurons. Inferred conditional ISI distributions and rate maps of our NPNR process in Fig. 2B are close to ground truth, showing the ability of our model to capture modulated spiking statistics drawn from various parametric families. To assess how well ISI statistics are captured, we apply time-rescaling using the GP posterior mean functions (Appendix A) which we visualize with quantile-quantile (QQ) plots [6] in Fig. 2C. Again, we see an excellent fit of our model compared to baseline models, indicating that only the NPNR is capable of satisfactorily capturing the empirical ISI statistics. Learned temporal kernel timescales of the NPNR process in Fig. 2B show a clear separation between the time since last spike τ dimension (lag 0) and lagging ISI ∆ dimensions (lag ≥ 1 ) with the dotted relevance boundary at l = 3 (dimensionless), as expected for renewal processes.

4.2 Neural data

Now we apply our method to head direction cells in freely moving mice [63, 64] and place cells in rats running along a linear track [54]. We select 33 units from the mouse and 35 units from the rat data, which leads to around 36 and 68 million data points to fit in the training set, respectively (see Appendix C for preprocessing details and Appendix B.2 on data scaling). Experiments involve fitting to the first half of a dataset ( ∼ 18 min. for mouse, ∼ 32 min. for rat), and testing on the second half split into 5 consecutive segments. The split into 5 test folds is used to quantify dataset variability. For

10 -8

10 -4

KS

p

-values

Poisson

gamma

inv. Gauss.

RC cond. P

RC cond. G

RC cond. IG

NP cond. P

NPNR (ours)

300

400

test ELL (nats/s)

0

quantile 1

0

1

EDF

log CIF

time

trials

1 s

1234

order

frequency

neuron index

10

0

10

1

lag 0

lag 1

lag 2

lag 3

density

2.5

5.0

average rate (Hz)

1

2

average CV

linear

GP

0

1

CV-rate

R

2

0

3

rate (Hz)

0

2

π

head direction

1

2

0

25

neuron 28

1

2

0

2

ISI (s)

probability

A

B

C

D

E

F

S

prediction, we evaluate the expected log likelihood summed over all neurons

using Gauss-Hermite quadrature with 50 points (Monte Carlo for renewal processes, see Appendix B). To assess goodness-of-fit to ISI statistics, we compute QQ plots as before and apply the KolmogorovSmirnov (KS) test, giving a p -value per neuron that indicates how likely the data came from the model (Appendix A). Baselines are the inhomogeneous Poisson (P), rate-rescaled gamma (G) and inverse Gaussian (IG) renewal [6, 83], raised cosine (RC) filter conditional Poisson [66, 85] and renewal [65], and nonparametric (NP) filter conditional Poisson processes [18] (details in Appendix C).

4.2.1 Mouse head direction cell data

Wechoose the animal head direction as our 1D input covariate x . From the KS test p -value distribution, we see that our model outperforms all baselines in capturing ISI statistics (Fig. 3A left, higher is better; Fig. 3B, QQ plots closer to diagonal). It performs competitively to conditional Poisson processes in terms of predictive performance (Fig. 3A right), but those models fail to capture ISI statistics. In addition, we note the spiking saturation in some samples of the nonparametric conditional Poisson model (Fig. 3C, purple) due to a known instability [3, 29]. Samples from our model (Fig. 3C, gray) exhibit visually similar spike patterns to the real spike train segment. Furthermore, kernel timescales in Fig. 3D show a sizable fraction of the population is characterized by a non-renewal spiking process.

Neural dispersion regimes From Fig. 3E and F, we observe both under- and overdispersion (CV smaller and bigger than one) consistent with a previous study based on spike counts [48]. Estimated instantaneous rates and CVs in Fig. 3E are computed using the conditional ISI distribution evaluated along the time series of covariates in the training data. One can regress instantaneous CV against rate, and the CV-rate R 2 shows some cells with near linear relations and some with nonlinear trends that can be captured by a GP. Despite that, many cells still show a low overall R 2 , implying there is no

10 -8

10 -4

KS

p

-values

Poisson

gamma

inv. Gauss.

RC cond. P

RC cond. G

RC cond. IG

NP cond. P

NPNR (ours)

-200

-100

0

test ELL (nats/s)

0

quantile 1

0

1

EDF

log CIF

time

trials

1 s

1234

order

frequency

neuron index

10

0

10

1

lag 0

lag 1

lag 2

lag 3

density

1

2

average rate (Hz)

1

2

average CV

linear

GP

0

1

CV-rate

R

2

2

π

neuron 2

neuron 16

0

rate

0

2.2

0.4

4.6

0.2

min

1

50 cm

0

2

ISI (s)

probability

A

B

C

D

E

F

x

position

LFP

θ

-phase

S

parametric relation. Most neurons increase CV with rate, while some show a slight linear decrease reminiscent of refractory Poisson processes (Eq. 31, see Fig. 7 for CV-rate patterns).

4.2.2 Rat place cell data

In this case, x is 3D consisting of the body position along the track, head direction, and local field potential (LFP) θ -phase. Our model significantly outperforms all baselines, having both a better KS test p -value distribution (Fig. 4A left, higher is better; Fig. 4B, QQ plots closer to diagonal) and predictive performance (Fig. 4A, right). Note that the ELLs for this dataset differ from those shown in Fig. 3A due to the fundamentally different (less predictable) spiking statistics of place cells compared to head direction cells (e.g. due to theta oscillations). As the nonparametric conditional Poisson model introduces nonparametric but covariate-independent variability patterns, these results highlights the importance of modeling covariate-dependent spiking variability. Note that the rate-rescaled renewal processes struggle to fit this data, with test ELLs of inverse Gaussian models below -200 nats/s (Fig. 4A right). Samples from our model (Fig. 4C, gray) show it captures the characteristic bursting nature of the real spike train segment. Kernel timescales in Fig. 4D show most cells are described well by a renewal process, different to mouse data Fig. 4D.

Capturing overdispersion We see CV values in Fig. 4E higher than the mouse thalamus dataset, consistent with overdispersion of place cell discharge in 2D open field navigation [26]. Similar to the mouse data, the CV-rate R 2 again shows there is generally no parametric relation. We also tend to observe larger increases in CV with firing rate compared to mouse data (Fig. 9).

θ -modulation and phase precession Spiking activity modulation during θ -cycles [53] is prominent in rat hippocampus, and is visible here in the log CIF (Fig. 4C). We also see phase precession [76] in Fig. 4F (top), a classical example where spike timing relative to some rhythm has coding significance [33]. Our model enables one to extract not only spiking intensity but also variability, and shows that variability generally inherits the phase precession pattern (Fig. 4F bottom).

5 Discussion

5.1 Limitations and further work

ISI statistics Apart from the coefficient of variation, there are other ISI statistics that characterize spiking dynamics aspects such as bursting or regularity. Of particular interest is the local coefficient of variation [75], which involves joint statistics of consecutive ISIs (∆ ( i ) , ∆ ( i -1) ) that can be computed

from our model (see Appendix D). The same applies to serial correlations [25], which may provide insights into biophysical details [77]. Furthermore, quantifying the shape of ISI distributions is of interest as it is associated with various properties of the underlying neural circuit dynamics [61].

Neural correlations To capture correlations in multivariate spike train data, direct spike couplings as in GLMs are less suitable for current neural recordings compared to latent variable models due to the sparse sampling of populations by electrodes [50]. Combining the latter alongside observed covariates [48] with our point process provides a powerful framework for capturing correlations [81], which can have significant impact on neural coding [56]. To perform goodness-of-fit tests, the Kolmogorov-Smirnov test with time-rescaling can be extended to the multivariate case [30, 92].

5.2 Conclusion and impact

We introduced the Bayesian nonparametric non-renewal (NPNR) process for flexible modeling of variability in neural spike train data. On synthetic renewal process data, NPNR successfully captures spiking statistics and their modulation by covariates, and finds renewal order in the spike-history dependence. When applied to mouse head direction cells and rat hippocampal place cells, NPNR has competitive or improved predictive performance to established baseline models, and is superior in terms of capturing ISI statistics, establishing the importance of capturing covariate-dependent variability. NPNR-based analyses recover known behavioral tuning, while also revealing novel patterns of spiking variability at millisecond timescales that are compatible with count-based studies.

Neural firing rates traditionally characterize most computational functions and information encoded by neurons [16, 17, 33], but recent work on V1 [20, 27, 36, 60] and hippocampal place cells [84] have started to assign computationally well-defined roles to variability in the context of representing uncertainty. Our method introduced in this paper is a principled tool for empirically characterizing neural spiking variability and its modulation at the timescales of individual spikes, and we hope our model will be useful for revealing new aspects of neural coding. Such findings are foundational to advances in computational and theoretical neuroscience, and may have downstream practical applications in designing and improving algorithms for brain-machine interfaces.

Acknowledgments and Disclosure of Funding

This work was supported by the Cambridge European and Wolfson College Scholarship by the Cambridge Trust (D.L.) and by the Wellcome Trust (Investigator Award in Science 212262/Z/18/Z to M.L.). We are grateful to Kristopher Jensen, Marine Schimel and Valentina Njaradi for helpful comments on the manuscript. We would also like to thank Alexander Terenin and Jonathan So for helpful discussions.

References

Avila-Akerberg, O. and Chacron, M. J. (2011). Nonrenewal spike train statistics: causes and functional consequences on neural coding. Experimental brain research , 210(3):353-371.

Barbieri, R., Quirk, M. C., Frank, L. M., Wilson, M. A., and Brown, E. N. (2001). Construction and analysis of non-poisson stimulus-response models of neural spiking activity. Journal of neuroscience methods , 105(1):25-37.

Bellec, G., Wang, S., Modirshanechi, A., Brea, J., and Gerstner, W. (2021). Fitting summary statistics of neural data with a differentiable spiking network simulator. Advances in Neural Information Processing Systems , 34:18552-18563.

Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q. (2018). JAX: composable transformations of Python+NumPy programs.

Brown, E. N., Barbieri, R., Eden, U. T., and Frank, L. M. (2003). Likelihood methods for neural spike train data analysis. Computational neuroscience: A comprehensive approach , pages 253-286.

Brown, E. N., Barbieri, R., Ventura, V., Kass, R. E., and Frank, L. M. (2002). The time-rescaling theorem and its application to neural spike train data analysis. Neural computation , 14(2):325-346.

Brown, E. N., Frank, L. M., Tang, D., Quirk, M. C., and Wilson, M. A. (1998). A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. Journal of Neuroscience , 18(18):7411-7425.

Chacron, M. J., Lindner, B., and Longtin, A. (2004). Noise shaping by interval correlations increases information transfer. Physical review letters , 92(8):080601.

Chacron, M. J., Longtin, A., and Maler, L. (2001). Negative interspike interval correlations increase the neuronal capacity for encoding time-dependent stimuli. Journal of Neuroscience , 21(14):5328-5343.

Chen, T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. (2018). Neural ordinary differential equations. In Advances in neural information processing systems , pages 6571-6583.

Churchland, A. K., Kiani, R., Chaudhuri, R., Wang, X.-J., Pouget, A., and Shadlen, M. N. (2011). Variance as a signature of neural computations during decision making. Neuron , 69(4):818-831.

Churchland, M. M., Byron, M. Y., Cunningham, J. P., Sugrue, L. P., Cohen, M. R., Corrado, G. S., Newsome, W. T., Clark, A. M., Hosseini, P., Scott, B. B., et al. (2010). Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nature neuroscience , 13(3):369.

Coleman, T. P. and Sarma, S. S. (2010). A computationally efficient method for nonparametric modeling of neural spiking activity with point processes. Neural Computation , 22(8):2002-2030.

Cox, D. R. (1972). The statistical analysis of dependencies in point processes. Stochastic Point Processes. Wiley: New York , pages 55-66.

Cunningham, J. P., Shenoy, K. V., and Sahani, M. (2008). Fast gaussian process methods for point process intensity estimation. In Proceedings of the 25th international conference on Machine learning , pages 192-199.

Cunningham, J. P., Yu, B. M., Shenoy, K. V., and Sahani, M. (2007). Inferring neural firing rates from spike trains using gaussian processes. Advances in neural information processing systems , 20.

DePasquale, B., Sussillo, D., Abbott, L., and Churchland, M. M. (2023). The centrality of population-level factors to network computation is demonstrated by a versatile approach for training spiking networks. Neuron .

Dowling, M., Zhao, Y., and Park, I. M. (2020). Non-parametric generalized linear model. arXiv preprint arXiv:2009.01362 .

Duncker, L. and Sahani, M. (2018). Temporal alignment and latent gaussian process factor inference in population spike trains. Advances in neural information processing systems , 31.

Echeveste, R., Aitchison, L., Hennequin, G., and Lengyel, M. (2020). Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference. bioRxiv , page 696088.

Ecker, A. S., Berens, P., Cotton, R. J., Subramaniyan, M., Denfield, G. H., Cadwell, C. R., Smirnakis, S. M., Bethge, M., and Tolias, A. S. (2014). State dependence of noise correlations in macaque primary visual cortex. Neuron , 82(1):235-248.

Engel, T. A., Helbig, B., Russell, D. F., Schimansky-Geier, L., and Neiman, A. B. (2009). Coherent stochastic oscillations enhance signal detection in spiking neurons. Physical Review E , 80(2):021919.

Escola, S., Fontanini, A., Katz, D., and Paninski, L. (2011). Hidden markov models for the stimulus-response relationships of multistate neural systems. Neural computation , 23(5):10711132.

Faisal, A. A., Selen, L. P., and Wolpert, D. M. (2008). Noise in the nervous system. Nature reviews neuroscience , 9(4):292-303.

Farkhooi, F., Strube-Bloss, M. F., and Nawrot, M. P. (2009). Serial correlation in neural spike trains: Experimental evidence, stochastic modeling, and single neuron variability. Physical Review E , 79(2):021905.

Fenton, A. A. and Muller, R. U. (1998). Place cell discharge is extremely variable during individual passes of the rat through the firing field. Proceedings of the National Academy of Sciences , 95(6):3182-3187.

Fiser, J., Berkes, P., Orbán, G., and Lengyel, M. (2010). Statistically optimal perception and learning: from behavior to neural representations. Trends in cognitive sciences , 14(3):119-130.

Gao, Y., Busing, L., Shenoy, K. V., and Cunningham, J. P. (2015). High-dimensional neural spike train analysis with generalized count linear dynamical systems. In Advances in neural information processing systems , pages 2044-2052.

Gerhard, F., Deger, M., and Truccolo, W. (2017). On the stability and dynamics of stochastic spiking neuron models: Nonlinear hawkes process and point process glms. PLoS computational biology , 13(2):e1005390.

Gerhard, F., Haslinger, R., and Pipa, G. (2011). Applying the multivariate time-rescaling theorem to neural population models. Neural computation , 23(6):1452-1483.

Gerstner, W. (2001). A framework for spiking neuron models: The spike response model. In Handbook of biological physics , volume 4, pages 469-516. Elsevier.

Gerstner, W., Kistler, W. M., Naud, R., and Paninski, L. (2014). Neuronal dynamics: From single neurons to networks and models of cognition . Cambridge University Press.

Gerstner, W., Kreiter, A. K., Markram, H., and Herz, A. V. (1997). Neural codes: firing rates and beyond. Proceedings of the National Academy of Sciences , 94(24):12740-12741.

Ghanbari, A., Lee, C. M., Read, H. L., and Stevenson, I. H. (2019). Modeling stimulusdependent variability improves decoding of population neural responses. Journal of Neural Engineering , 16(6):066018.

Harris, K. D., Henze, D. A., Hirase, H., Leinekugel, X., Dragoi, G., Czurkó, A., and Buzsáki, G. (2002). Spike train dynamics predicts theta-related phase precession in hippocampal pyramidal cells. Nature , 417(6890):738-741.

Hennequin, G., Ahmadian, Y., Rubin, D. B., Lengyel, M., and Miller, K. D. (2018). The dynamical regime of sensory cortex: stable dynamics around a single stimulus-tuned attractor account for patterns of noise variability. Neuron , 98(4):846-860.

Hensman, J., Fusi, N., and Lawrence, N. D. (2013). Gaussian processes for big data. In Nicholson, A. E. and Smyth, P., editors, Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, UAI 2013, Bellevue, WA, USA, August 11-15, 2013 . AUAI Press.

Hill, D. N., Mehta, S. B., and Kleinfeld, D. (2011). Quality metrics to accompany spike sorting of extracellular signals. Journal of Neuroscience , 31(24):8699-8705.

Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J. (2013). Stochastic variational inference. The Journal of Machine Learning Research , 14(1):1303-1347.

Jensen, K., Kao, T.-C., Stone, J., and Hennequin, G. (2021). Scalable bayesian gpfa with automatic relevance determination and discrete noise models. Advances in Neural Information Processing Systems , 34:10613-10626.

Jia, J. and Benson, A. R. (2019). Neural jump stochastic differential equations. Advances in Neural Information Processing Systems , 32.

Kass, R. E., Kelly, R. C., and Loh, W.-L. (2011). Assessment of synchrony in multiple neural spike trains using loglinear point process models. The annals of applied statistics , 5(2B):1262.

Kass, R. E. and Ventura, V. (2001). A spike-train probability model. Neural computation , 13(8):1713-1720.

Kayser, C., Logothetis, N. K., and Panzeri, S. (2010). Millisecond encoding precision of auditory cortex neurons. Proceedings of the National Academy of Sciences , 107(39):16976-16981.

Kim, S., Putrino, D., Ghosh, S., and Brown, E. N. (2011). A granger causality measure for point process models of ensemble neural spiking activity. PLoS computational biology , 7(3):e1001110.

Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 .

Linderman, S. W., Adams, R. P., and Pillow, J. W. (2015). Inferring structured connectivity from spike trains under negative-binomial generalized linear models. Cosyne Abstracts .

Liu, D. and Lengyel, M. (2021). A universal probabilistic spike count model reveals ongoing modulation of neural variability. Advances in Neural Information Processing Systems , 34:1339213405.

London, M., Roth, A., Beeren, L., Häusser, M., and Latham, P. E. (2010). Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature , 466(7302):123127.

Macke, J. H., Buesing, L., Cunningham, J. P., Yu, B. M., Shenoy, K. V., and Sahani, M. (2011). Empirical models of spiking in neural populations. Advances in neural information processing systems , 24.

Mei, H. and Eisner, J. M. (2017). The neural hawkes process: A neurally self-modulating multivariate point process. Advances in neural information processing systems , 30.

Mensi, S., Naud, R., and Gerstner, W. (2011). From stochastic nonlinear integrate-and-fire to generalized linear models. Advances in Neural Information Processing Systems , 24.

Mizuseki, K., Sirota, A., Pastalkova, E., and Buzsáki, G. (2009). Theta oscillations provide temporal windows for local circuit computation in the entorhinal-hippocampal loop. Neuron , 64(2):267-280.

Mizuseki, K., Sirota, A., Pastalkova, E., Diba, K., and Buzsáki, G. (2013). Multiple single unit recordings from different rat hippocampal and entorhinal regions while the animals were performing multiple behavioral tasks. CRCNS org .

Møller, J., Syversveen, A. R., and Waagepetersen, R. P. (1998). Log gaussian cox processes. Scandinavian journal of statistics , 25(3):451-482.

Moreno-Bote, R., Beck, J., Kanitscheider, I., Pitkow, X., Latham, P., and Pouget, A. (2014). Information-limiting correlations. Nature neuroscience , 17(10):1410.

Nagele, J., Herz, A. V., and Stemmler, M. B. (2020). Untethered firing fields and intermittent silences: Why grid-cell discharge is so variable. Hippocampus .

Nawrot, M. P., Boucsein, C., Molina, V. R., Riehle, A., Aertsen, A., and Rotter, S. (2008). Measurement of variability dynamics in cortical spike trains. Journal of neuroscience methods , 169(2):374-390.

Omi, T., Ueda, N., and Aihara, K. (2019). Fully neural network based model for general temporal point processes. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc.

Orbán, G., Berkes, P., Fiser, J., and Lengyel, M. (2016). Neural variability and sampling-based probabilistic representations in the visual cortex. Neuron , 92(2):530-543.

Ostojic, S. (2011). Interspike interval distributions of spiking neurons driven by fluctuating inputs. Journal of neurophysiology , 106(1):361-373.

Paninski, L. (2004). Maximum likelihood estimation of cascade point-process neural encoding models. Network: Computation in Neural Systems , 15(4):243-262.

Peyrache, A. and Buzsáki, G. (2015). Extracellular recordings from multi-site silicon probes in the anterior thalamus and subicular formation of freely moving mice. CRCNS .

Peyrache, A., Lacroix, M. M., Petersen, P. C., and Buzsáki, G. (2015). Internally organized mechanisms of the head direction sense. Nature neuroscience , 18(4):569-575.

Pillow, J. W. (2009). Time-rescaling methods for the estimation and assessment of non-poisson neural encoding models. In Advances in neural information processing systems , pages 1473-1481.

Pillow, J. W., Shlens, J., Paninski, L., Sher, A., Litke, A. M., Chichilnisky, E., and Simoncelli, E. P. (2008). Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature , 454(7207):995-999.

Ponce-Alvarez, A., Thiele, A., Albright, T. D., Stoner, G. R., and Deco, G. (2013). Stimulusdependent variability and noise correlations in cortical mt neurons. Proceedings of the National Academy of Sciences , 110(32):13162-13167.

Rad, K. R. and Paninski, L. (2010). Efficient, adaptive estimation of two-dimensional firing rate surfaces via gaussian process methods. Network: Computation in Neural Systems , 21(3-4):142168.

Ratnam, R. and Nelson, M. E. (2000). Nonrenewal statistics of electrosensory afferent spike trains: implications for the detection of weak sensory signals. Journal of Neuroscience , 20(17):6672-6683.

Reich, D. S., Victor, J. D., Knight, B. W., Ozaki, T., and Kaplan, E. (1997). Response variability and timing precision of neuronal spike trains in vivo. Journal of neurophysiology , 77(5):28362841.

Sahin, I. (1993). A generalization of renewal processes. Operations research letters , 13(4):259263.

Shchur, O., Gao, N., Biloš, M., and Günnemann, S. (2020). Fast and flexible temporal point processes with triangular maps. Advances in Neural Information Processing Systems , 33:73-84.

Shimazaki, H. and Shinomoto, S. (2007). A method for selecting the bin size of a time histogram. Neural computation , 19(6):1503-1527.

Shimokawa, T., Koyama, S., and Shinomoto, S. (2010). A characterization of the time-rescaled gamma process as a model for spike trains. Journal of computational neuroscience , 29(1-2):183191.

Shinomoto, S., Miura, K., and Koyama, S. (2005). A measure of local variation of inter-spike intervals. Biosystems , 79(1-3):67-72.

Skaggs, W. E., McNaughton, B. L., Wilson, M. A., and Barnes, C. A. (1996). Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus , 6(2):149-172.

Song, S., Lee, J. A., Kiselev, I., Iyengar, V ., Trapani, J. G., and Tania, N. (2018). Mathematical modeling and analyses of interspike-intervals of spontaneous activity in afferent neurons of the zebrafish lateral line. Scientific Reports , 8(1):1-14.

Stein, R. B., Gossen, E. R., and Jones, K. E. (2005). Neuronal variability: noise or part of the signal? Nature Reviews Neuroscience , 6(5):389-397.

Teh, Y. W. and Rao, V. (2011). Gaussian process modulated renewal processes. In Advances in Neural Information Processing Systems , pages 2474-2482.

Titsias, M. and Lawrence, N. D. (2010). Bayesian gaussian process latent variable model. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 844-851.

Truccolo, W. (2016). From point process observations to collective neural dynamics: Nonlinear hawkes process glms, low-dimensional dynamics and coarse graining. Journal of Physiology-Paris , 110(4):336-347.

Truccolo, W. and Donoghue, J. P. (2007). Nonparametric modeling of neural point processes via stochastic gradient boosting regression. Neural computation , 19(3):672-705.

Truccolo, W., Eden, U. T., Fellows, M. R., Donoghue, J. P., and Brown, E. N. (2005). A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. Journal of neurophysiology , 93(2):1074-1089.

Ujfalussy, B. B. and Orbán, G. (2022). Sampling motion trajectories during hippocampal theta sequences. Elife , 11:e74058.

Weber, A. I. and Pillow, J. W. (2017). Capturing the dynamical repertoire of single neurons with generalized linear models. Neural computation , 29(12):3260-3289.

Williams, A. H., Poole, B., Maheswaranathan, N., Dhawale, A. K., Fisher, T., Wilson, C. D., Brann, D. H., Trautmann, E. M., Ryu, S., Shusterman, R., et al. (2020). Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping. Neuron , 105(2):246-259.

Williams, C. K. and Rasmussen, C. E. (2006). Gaussian processes for machine learning , volume 2. MIT press Cambridge, MA.

Wilson, J., Borovitskiy, V., Terenin, A., Mostowsky, P., and Deisenroth, M. (2020). Efficiently sampling functions from gaussian process posteriors. In International Conference on Machine Learning , pages 10292-10302. PMLR.

Wilson, J. T., Borovitskiy, V ., Terenin, A., Mostowsky, P., and Deisenroth, M. P. (2021). Pathwise conditioning of gaussian processes. The Journal of Machine Learning Research , 22(1):4741-4787.

Xiao, S., Yan, J., Yang, X., Zha, H., and Chu, S. (2017). Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 31.

Yang, R., Gupta, G., and Bogdan, P. (2019). Data-driven perception of neuron point process with unknown unknowns. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems , pages 259-269.

Yousefi, A., Amidi, Y., Nazari, B., and Eden, U. T. (2020). Assessing goodness-of-fit in marked point process models of neural population coding via time and rate rescaling. Neural Computation , 32(11):2145-2186.

Yu, B. M., Cunningham, J. P., Santhanam, G., Ryu, S., Shenoy, K. V., and Sahani, M. (2008). Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. Advances in neural information processing systems , 21:1881-1888.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce the Bayesian nonparametric non-renewal (NPNR) process to model variability in neural spike trains with covariate dependence. The method generalizes modulated renewal processes using sparse variational Gaussian processes. Tested on synthetic data, as well as mouse head direction cell data and rat hippocampal place cell data, NPNR shows its superiority in terms of capturing interspike interval statistics and predictive power.\n\nSTRENGTHS:\n+ Bayesian nonparametric non-renewal (NPNR) process to model variability in neural spike trains;\n+ Validation of the approach on synthetic data and mouse head direction cell data and rat hippocampal place cell data;\n\nWEAKNESSES:\n- There are some aspects that while they may be hard to analyze theoretically at least in the experimental part can be investigated through simulations. For example, how is the inference affected by the dimension of x_t, number of unmeasured or unknown neurons  that can act as perturbations, the number of samples and number of units/channels? I.e., how much data (in space and time) is needed for an efficient inference?\n- Within the experimental investigation, it is unclear what ELL values can be considered as good predictions. For example, if NPNR is used for k-step ahead forecasts, how many steps can it achieve with at least 50% accuracy?\n- Although this is a minor issue, the reader has some difficulties at seeing all the plots well from Figure 2 in particular the ISI probability plots. I assume they fit well gamma or exponential distribution.\n- A major issue is to check the literature and in unbiased way provide an accurate description even if the problems considered by prior work have a different or more advanced setup.\n\nQUESTIONS:\n1. The authors mention that the variational inference framework is scalable. In the synthetic and real-world experiments, the number of neurons/units are relatively small (less than 50). I wonder if there's any limitation of the method to scale up to larger ensemble systems with much more neurons?\n2. How does the proposed model perform in terms of predictive power when applied to synthetic data?\n3. The experiments on mouse head direction cell data and rat hippocampal place cell data shows the predictive performance of NPNR and baseline models by showing the expected log-likelihood. For the two datasets, the range of ELL for the models vary a lot and is hard to interpret. I wonder what value of ELL can be considered as a good prediction? For example, if NPNR is used for k-step ahead forecasts, how many steps can it achieve with at least 50% accuracy?\n4. The manuscript states that “Extending point process models with input-dependent variability has not been widely explored…” Multivariate auto-regressive frameworks and multiple covariates based models have been considered in \"A Granger causality measure for point process models of ensemble neural spiking activity.\" PLoS computational biology 7, no. 3 (2011): e1001110. \"Data-driven perception of neuron point process with unknown unknowns.\" In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, pp. 259-269. 2019. \"Variance as a signature of neural computations during decision making.\" Neuron 69, no. 4 (2011): 818-831. In general the prior work needs to be more exhaustively checked and discussed, as of now it is biased and solely based on one group while there are similar and related works from other groups.\n5. Within the context of multiple neuronal recordings there is always the issue of interference and the problem that we cannot with certainty measure exactly N number of neurons. The activity of N neurons may be influenced by another P neurons so the question is how we can subtract the effect or perturbations in order to accurately model the N neurons and their covariates, etc. This again has been tackled in the neuroscience literature and the authors should check this related problem of understanding neural computations with unknown influences.\n6. In the experiments, 1-D, 2-D and 3-D x_t are considered for the NPNR modeling. I wonder if and how the inference can be affected by the dimension of x_t, number of samples and number of units/channels? I.e., how much data (in space and time) is needed for an efficient inference?\n\nLIMITATIONS:\nNot applicable in my opinion, this is a mathematical modeling paper with applications in neuroscience.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the Bayesian nonparametric non-renewal process (NPNR) for inferencing both neural spiking intensity and variability. The tuning curve is based on a sparse variational Gaussian process (GP) prior, considering both spatial and temporal factors. They compare NPNR with other competitors on a synthetic dataset showing the capability of NPNR in inferencing the rate map and renewal density. On the two real-world neural datasets, they show that NPNR outperforms lots of competitors in terms of event prediction and interspike interval (ISI) recovery by different statistics combined with visualizations.\n\nSTRENGTHS:\n* Clear logical flow and presentation. Key maths are derived elegantly with lots of necessary details in the Appendix.\n* Both synthetic and real-world experiments are good and solid, and also supported by the code.\n* The literature review by the authors are exhaustive so that the comparison between different kind of models are clear and in detail.\n* The idea is new and intuitive, both preserve the interpretability and are not too simple.\n\nWEAKNESSES:\n* I'm very excited when seeing the model part. But when I get to Section 3.2, I feel a bit pity that we still need to do time discretization (convert the continuous timestamps TPP data to spike counts in time bins).\n\nQUESTIONS:\n* I think Eq. 15 should be $=$ rather than $\\propto$.\n* I'm wondering if this model can report a predictive log-likelihood using the mean as the estimation for the intensity in each time bin. In such a case, we can compare this model with other models (especially the simple GLM) to show that the proposed NPNR outperforms GLM? I'm expecting that this model will be slow but if the firing rates (tuning curve) recovery is better than GLM, the predictive log-likelihood (which is actually a golden criterion in neural latent variable models) should be better.\n* Can this model get information on the causal relationships between neurons? Is this model only dependent on time $t$ and external input $\\boldsymbol x$, but does not consider influences between neurons? From my understanding, this is not a latent variable model doing information extraction from coupled neurons (like dimensionality reduction), but getting the firing rate for each neuron, and the firing rate is mainly affected by the neuron itself and the external input $\\boldsymbol x$.\n\nLIMITATIONS:\n/", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

comments: SUMMARY:
The authors proposed a scalable Bayesian approach which generalizes modulated renewal processes using sparse variational Gaussian processes. They applied the proposed method to simulated and two real neural datasets and showed that the proposed method is effective on these datasets and outperforms other baseline methods.

STRENGTHS:
The paper is well written. The authors have done extensive experiments to show the effectiveness of the proposed method.

WEAKNESSES:
The proposed method doesn't incorporate any latent variables in the model. (see more in the questions section below.)

QUESTIONS:
- Method section: all the formulas are described for 1 neuron. It might be clearer to write likelihood and loss function in terms of multiple neurons.

- I wonder if the authors have done any comparisons to the parametric methods? e.g. Gao Y*, Archer E*, Paninski L, Cunningham JP (2016) Linear dynamical neural population models through nonlinear embeddings. NIPS 2016.

- The proposed method doesn't incorporate any latent variables. It might be worth adding the latents to discover useful representations from the data and fit the data variability better. I wonder if the model would fit data worse if miss one covariate in the inputs? (e.g. only includes location and direction in covariate not theta phase in the hippocampus data.) I feel that adding latents might help with this as well.

LIMITATIONS:
The authors have discussed the limitations and future work of their proposed method.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe variability of neural data is widely observed in many neuroscience experiments. Using statistical model to capture the variability structure plays an essential role in understanding neural computations. Generally, the variability of neural data is a result of non-stationary activities and dependencies on behavioral covariates. To tackle these challenges, the authors proposes a scalable Bayesian approach generalizing modulated renewal processes (NPNR) to analyze neural data variability. They develops a nonparametric generalization of modulated renewal processes beyond renewal order, which makes their method flexibly model irreducible “intrinsic” neural stochasticity and input-dependent variability. Furthermore, the authors apply stochastic variational inference and can fit the model to long time neural data given cubic time complexity in the number of inducing points. The performace of NPNR is evaluated on both synthetic and real neural datasets.\n\nSTRENGTHS:\n* The proposed method can model two types of neural variability: (1) capturing spiking statistics from non-stationary data; (2) capturing modulation by behavioral covariates.\n* To achieve the desired non-stationarity, the proposed method uses time warping on $\\tau$, which avoids the use of non-stationary kernls and maintains the ability to draw samples by pathwise conditioning.\n* The proposed inference method provides an elegant approach to determine the spike-history dependence in ISI statistics.\n* The authors' exposition of their motivation, contribution, and conclusions from the experiments are comprehensive and clear.\n* The proposed method would provide an important set of contributions to the field of neural coding.\n\nWEAKNESSES:\nThe proposed method is clearly written and well supported by experiments. I have nothing further to add here.\n\nQUESTIONS:\n* The proposed method captures ISI statistics using a spatio-temporal GP prior over CIF. Could the Neural Temporal Point Process (NTPP) perform similarly to your method in capturing ISI statistics? The CIF in NTPP is usually modeled by neural networks, and could this be more powerful to represent ISI distributions and capture ISI statistics?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a Bayesian nonparametric approach using modulated renewal process to model neural spike train, and capable of modeling the covariability. The method includes a nonparametric priors on conditional interspike interval distribution and automatic relevance determination for lagging interspike interval based on renewal order. The method is evaluated on one synthetic data and two real datasets on animal navigation. It demonstrates better performance than the current SOTA baselines in its capability of capturing the interspike interval statistics.\n\nSTRENGTHS:\n1. Motivation: the paper is well-motivated, modeling the spike train statistics is an important question, and the interspike interval statistics is an important property of neural dynamics, and potentially leads to identification of cell types, and functionality, etc. Designing a model that captures this property well is important.\n\n2. Method: it uses Bayesian nonparameteric approach that could fit complex data structures and patterns well, and it could infer the spike-history dependence using a data-driven approach.\n\n3. Results: the method demonstrate better accuracy than the current SOTA methods in multiple tasks and datasets.\n\nWEAKNESSES:\n1. Method: the model requires hyperparameter tuning of critical components includes $\\tau_w$  $K$, which might be hard to optimize, given its variability across neurons and datasets.\n\n2. Evaluation: the scalability of the method is a major concerns, as the datasets evaluated in this paper only has 9 neurons, ~30 units in each datasets. It's important to show how well the model performs in larger neural datasets.\n\n3. Complexity and computational cost: add evaluations based on the cost and speed of the proposed model and other baselines.\n\nQUESTIONS:\n1. Give a detailed introduction about the parameters that would be optimized under eqn 18. and list other hyper-parameters for reproducibility. \n\n2. Evaluate the method on larger scale neural datasets.\n\nLIMITATIONS:\n1. There is no potential negative societal impact of their work.\n2. The limitations about scalability of the proposed approach should be carefully addressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}