SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $n$  indexes neurons (of which there are  $N$ ) $^2$ ,  $\pmb{u} = [u_{1},\dots,u_{M}]$  denotes the set of  $M$  inducing points,  $p(\pmb{u})$  the GP prior at inducing locations,  $q(\pmb{u})$  the variational posterior, and  $q(\pmb{f}|\pmb{u})$  the conditional posterior (see Appendix B.1 for details). Combined with temporal mini-batching to fit batch segments of length  $T$ , we can fit to very long time series given the  $O\big(NTM^2 +NM^3\big)$  computational complexity. We also no longer rely on computing hazard functions of parametric renewal densities to obtain the CIF, which can be numerically unstable. Modulated renewal processes instead rely on a specialized thinning procedure [79], but we take a more scalable and general variational approach. Overall, we optimize the kernel hyperparameters, variational posterior mean and covariance, inducing point locations, and mean function parameters  $a_{m}$ ,  $b_{m}$  and  $\tau_{m}$  using gradient descent with Adam [46] (see Appendix C for details). The time warping parameter  $\tau_w$  is fixed in our experiments to the empirical mean ISI, and the hyperparameter  $K$  is fixed and chosen in advance (see also subsection on automatic relevance determination below).
Automatic relevance determination The Bayesian framework with GPs enables us to perform automatic relevance determination (ARD) over the input dimensions to automatically select relevant input [37, 80]. Applied to lagging ISI dimensions in our NPNR model, this provides an elegant approach to making a data-driven renewal assumption and generally determining the spike-history dependence of the CIF. We choose to fix  $\tau_w$  to the empirical mean ISI as shown in Fig. 1B (rather than learning it) to achieve interpretability of kernel timescales for ARD (Fig. 6A) at a small cost of performance (Fig. 12). For a chosen maximum lag  $K$ , there is no need for manual selection of the history interaction window as for GLM spike-history filters, though recent work on nonparametric GLM filters provides a related window size selection procedure [18]. In the spirit of Bayesian models, we choose  $K$  to give a sufficiently high capacity model [40, 80] to be able to flexibly capture history dependence, as seen in panel D of Fig. 3 and Fig. 4.
All datasets discretize spike trains and input time series at regular intervals of  $\Delta t = 1$  ms. We use a product kernel for  $k(\pmb{x},\pmb{x}^{\prime})$  with periodic kernels for angular dimensions, and squared exponential kernels in other cases. For  $k(\tilde{\tau},\tilde{\tau}^{\prime})$  and  $k(\tilde{\Delta},\tilde{\Delta}^{\prime})$ , we pick a product kernel with Matérn-3/2 (see Fig. 12 for different kernel choices) and set the maximum ISI lag  $K = 3$ . For illustration, conditional
Note the simple summation over  $n$  in Eq. 18, as our model does not capture neural correlations without introducing latent covariates [48]. In other words, in its current form, our model treats neurons as independent.
Figure 2: Validation on synthetic data. (A) True rate maps (brighter is higher) defined over a square environment (top) and base renewal densities (bottom) in each column given by gamma (left), log normal (middle) and inverse Gaussian (right) distributions with various shape parameters. Each color corresponds to a separate neuron. (B) Posterior mean rate maps (top left) and conditional ISI distribution samples in gray overlaid on true renewal densities at various locations (bottom) for the NPNR process fit to synthetic data. The relevance boundary (dotted line) for kernel timescales (top right) is placed at  $l = 3$  (dimensionless). (C) QQ-plots of fitted models (each curve is a neuron).
ISI distributions and corresponding tuning curves are computed by fixing  $\Delta_{k}$  to be the mean ISI per neuron. Firing rates are defined as  $1 / \mathbb{E}[\tau]$ , since this corresponds to the number of spikes fired per unit time in infinitely large time bins for a renewal process (Eq. 26). GP inducing points were randomly initialized, and for a fair comparison, all models used 8 inducing points for each covariate dimension (including temporal dimensions  $\tau$  and  $\Delta$  in the NPNR process). For each experiment, we repeat model fitting with 3 different random seeds and pick the model with the best training likelihood. Further details on experiments are presented in Appendix C.
4.1 Validation on synthetic data
For validating our approach, we generate  $1000\mathrm{s}$  of data using rate-rescaling [65] mimicking a place cell population of 9 neurons for an animal moving in a 2D square arena, each with a unique rate map and renewal density (Fig. 2A, details in Appendix C). The models applied are baseline Poisson, raised cosine filter conditional Poisson and rate-rescaled gamma processes (Appendix C), and our NPNR process. Note that the rescaled gamma process is within-model class for 3 of the synthetic neurons. Inferred conditional ISI distributions and rate maps of our NPNR process in Fig. 2B are close to ground truth, showing the ability of our model to capture modulated spiking statistics drawn from various parametric families. To assess how well ISI statistics are captured, we apply time-rescaling using the GP posterior mean functions (Appendix A) which we visualize with quantile-quantile (QQ) plots [6] in Fig. 2C. Again, we see an excellent fit of our model compared to baseline models, indicating that only the NPNR is capable of satisfactorily capturing the empirical ISI statistics. Learned temporal kernel timescales of the NPNR process in Fig. 2B show a clear separation between the time since last spike  $\tau$  dimension (lag 0) and lagging ISI  $\Delta$  dimensions (lag  $\geq 1$ ) with the dotted relevance boundary at  $l = 3$  (dimensionless), as expected for renewal processes.
4.2 Neural data
Now we apply our method to head direction cells in freely moving mice [63, 64] and place cells in rats running along a linear track [54]. We select 33 units from the mouse and 35 units from the rat data, which leads to around 36 and 68 million data points to fit in the training set, respectively (see Appendix C for preprocessing details and Appendix B.2 on data scaling). Experiments involve fitting to the first half of a dataset ( $\sim 18$  min. for mouse,  $\sim 32$  min. for rat), and testing on the second half split into 5 consecutive segments. The split into 5 test folds is used to quantify dataset variability. For
Figure 3: Application to mouse head direction cell data. (A) Violin plot of KS  $p$ -values per neuron (left, lines marking quartiles) and test expected log likelihoods with errorbars showing s.e.m. across test folds (right). Larger values in both metrics indicate better model fit to data. (B) QQ-plots for various models (each curve is a neuron) identified by color (panel A left). (C) Predicted log CIF (middle) for an observed spike train (top) and posterior spike train samples (bottom) conditioned on the same covariates  $x_{t}$  for various models identified by color. (D) Left: temporal kernel timescales for  $\tau$  (lag 0) and  $\Delta_{k}$  dimensions with the relevance boundary at  $l = 3$  (dimensionless). Right: histogram of "ISI-order" (1 + largest lag  $k$  for which  $k$  is below the boundary) across neurons. (E) Time average of estimated instantaneous rates and CVs from the training data (left) and  $R^2$  values of CV-rate regression with a linear and a GP model (right). (F) Posterior median and 95% intervals of tuning curves over head direction for the rate and CV, with posterior ISI distribution samples (right) at dashed locations.
prediction, we evaluate the expected log likelihood summed over all neurons
$$
\operatorname {E L L} = \sum_ {n} \mathbb {E} _ {q (\boldsymbol {f} _ {n})} [ \log p (\boldsymbol {y} _ {n} | \boldsymbol {f} _ {n}) ] \tag {19}
$$
using Gauss-Hermite quadrature with 50 points (Monte Carlo for renewal processes, see Appendix B). To assess goodness-of-fit to ISI statistics, we compute QQ plots as before and apply the Kolmogorov-Smirnov (KS) test, giving a  $p$ -value per neuron that indicates how likely the data came from the model (Appendix A). Baselines are the inhomogeneous Poisson (P), rate-rescaled gamma (G) and inverse Gaussian (IG) renewal [6, 83], raised cosine (RC) filter conditional Poisson [66, 85] and renewal [65], and nonparametric (NP) filter conditional Poisson processes [18] (details in Appendix C).
4.2.1 Mouse head direction cell data
We choose the animal head direction as our 1D input covariate  $x$ . From the KS test  $p$ -value distribution, we see that our model outperforms all baselines in capturing ISI statistics (Fig. 3A left, higher is better; Fig. 3B, QQ plots closer to diagonal). It performs competitively to conditional Poisson processes in terms of predictive performance (Fig. 3A right), but those models fail to capture ISI statistics. In addition, we note the spiking saturation in some samples of the nonparametric conditional Poisson model (Fig. 3C, purple) due to a known instability [3, 29]. Samples from our model (Fig. 3C, gray) exhibit visually similar spike patterns to the real spike train segment. Furthermore, kernel timescales in Fig. 3D show a sizable fraction of the population is characterized by a non-renewal spiking process.
Neural dispersion regimes From Fig. 3E and F, we observe both under- and overdispersion (CV smaller and bigger than one) consistent with a previous study based on spike counts [48]. Estimated instantaneous rates and CVs in Fig. 3E are computed using the conditional ISI distribution evaluated along the time series of covariates in the training data. One can regress instantaneous CV against rate, and the CV-rate  $R^2$  shows some cells with near linear relations and some with nonlinear trends that can be captured by a GP. Despite that, many cells still show a low overall  $R^2$ , implying there is no
Figure 4: Application to rat hippocampal place cell data. (A)-(E) Similar to Fig. 3A-E. (F) Posterior mean tuning maps over  $x$ -position and  $\theta$ -phase for the rate and CV (left) for left-to-right runs (based on head direction) with posterior ISI distribution samples (right) at marked locations.
parametric relation. Most neurons increase CV with rate, while some show a slight linear decrease reminiscent of refractory Poisson processes (Eq. 31, see Fig. 7 for CV-rate patterns).
4.2.2 Rat place cell data
In this case,  $\pmb{x}$  is 3D consisting of the body position along the track, head direction, and local field potential (LFP)  $\theta$ -phase. Our model significantly outperforms all baselines, having both a better KS test  $p$ -value distribution (Fig. 4A left, higher is better; Fig. 4B, QQ plots closer to diagonal) and predictive performance (Fig. 4A, right). Note that the ELLs for this dataset differ from those shown in Fig. 3A due to the fundamentally different (less predictable) spiking statistics of place cells compared to head direction cells (e.g. due to theta oscillations). As the nonparametric conditional Poisson model introduces nonparametric but covariate-independent variability patterns, these results highlights the importance of modeling covariate-dependent spiking variability. Note that the rate-rescaled renewal processes struggle to fit this data, with test ELLs of inverse Gaussian models below -200 nats/s (Fig. 4A right). Samples from our model (Fig. 4C, gray) show it captures the characteristic bursting nature of the real spike train segment. Kernel timescales in Fig. 4D show most cells are described well by a renewal process, different to mouse data Fig. 4D.
Capturing overdispersion We see CV values in Fig. 4E higher than the mouse thalamus dataset, consistent with overdispersion of place cell discharge in 2D open field navigation [26]. Similar to the mouse data, the CV-rate  $R^2$  again shows there is generally no parametric relation. We also tend to observe larger increases in CV with firing rate compared to mouse data (Fig. 9).
$\theta$ -modulation and phase precession Spiking activity modulation during  $\theta$ -cycles [53] is prominent in rat hippocampus, and is visible here in the log CIF (Fig. 4C). We also see phase precession [76] in Fig. 4F (top), a classical example where spike timing relative to some rhythm has coding significance [33]. Our model enables one to extract not only spiking intensity but also variability, and shows that variability generally inherits the phase precession pattern (Fig. 4F bottom).
5.1 Limitations and further work
ISI statistics Apart from the coefficient of variation, there are other ISI statistics that characterize spiking dynamics aspects such as bursting or regularity. Of particular interest is the local coefficient of variation [75], which involves joint statistics of consecutive ISIs  $(\Delta^{(i)}, \Delta^{(i-1)})$  that can be computed
from our model (see Appendix D). The same applies to serial correlations [25], which may provide insights into biophysical details [77]. Furthermore, quantifying the shape of ISI distributions is of interest as it is associated with various properties of the underlying neural circuit dynamics [61].
Neural correlations To capture correlations in multivariate spike train data, direct spike couplings as in GLMs are less suitable for current neural recordings compared to latent variable models due to the sparse sampling of populations by electrodes [50]. Combining the latter alongside observed covariates [48] with our point process provides a powerful framework for capturing correlations [81], which can have significant impact on neural coding [56]. To perform goodness-of-fit tests, the Kolmogorov-Smirnov test with time-rescaling can be extended to the multivariate case [30, 92].
5.2 Conclusion and impact
We introduced the Bayesian nonparametric non-renewal (NPNR) process for flexible modeling of variability in neural spike train data. On synthetic renewal process data, NPNR successfully captures spiking statistics and their modulation by covariates, and finds renewal order in the spike-history dependence. When applied to mouse head direction cells and rat hippocampal place cells, NPNR has competitive or improved predictive performance to established baseline models, and is superior in terms of capturing ISI statistics, establishing the importance of capturing covariate-dependent variability. NPNR-based analyses recover known behavioral tuning, while also revealing novel patterns of spiking variability at millisecond timescales that are compatible with count-based studies.
Neural firing rates traditionally characterize most computational functions and information encoded by neurons [16, 17, 33], but recent work on V1 [20, 27, 36, 60] and hippocampal place cells [84] have started to assign computationally well-defined roles to variability in the context of representing uncertainty. Our method introduced in this paper is a principled tool for empirically characterizing neural spiking variability and its modulation at the timescales of individual spikes, and we hope our model will be useful for revealing new aspects of neural coding. Such findings are foundational to advances in computational and theoretical neuroscience, and may have downstream practical applications in designing and improving algorithms for brain-machine interfaces.
This work was supported by the Cambridge European and Wolfson College Scholarship by the Cambridge Trust (D.L.) and by the Wellcome Trust (Investigator Award in Science 212262/Z/18/Z to M.L.). We are grateful to Kristopher Jensen, Marine Schimel and Valentina Njaradi for helpful comments on the manuscript. We would also like to thank Alexander Terenin and Jonathan So for helpful discussions.
[1] Avila-Akerberg, O. and Chacron, M. J. (2011). Nonrenewal spike train statistics: causes and functional consequences on neural coding. Experimental brain research, 210(3):353-371.
[2] Barbieri, R., Quirk, M. C., Frank, L. M., Wilson, M. A., and Brown, E. N. (2001). Construction and analysis of non-poisson stimulus-response models of neural spiking activity. Journal of neuroscience methods, 105(1):25-37.
[3] Bellec, G., Wang, S., Modirshanechi, A., Brea, J., and Gerstner, W. (2021). Fitting summary statistics of neural data with a differentiable spiking network simulator. Advances in Neural Information Processing Systems, 34:18552-18563.
[4] Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q. (2018). JAX: composable transformations of Python+NumPy programs.
[5] Brown, E. N., Barbieri, R., Eden, U. T., and Frank, L. M. (2003). Likelihood methods for neural spike train data analysis. Computational neuroscience: A comprehensive approach, pages 253-286.
[81] Truccolo, W. (2016). From point process observations to collective neural dynamics: Nonlinear hawkes process glms, low-dimensional dynamics and coarse graining. Journal of Physiology-Paris, 110(4):336-347.
[82] Truccolo, W. and Donoghue, J. P. (2007). Nonparametric modeling of neural point processes via stochastic gradient boosting regression. Neural computation, 19(3):672-705.
[83] Truccolo, W., Eden, U. T., Fellows, M. R., Donoghue, J. P., and Brown, E. N. (2005). A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. Journal of neurophysiology, 93(2):1074-1089.
[84] Ujfalussy, B. B. and Orbán, G. (2022). Sampling motion trajectories during hippocampal theta sequences. *Elife*, 11:e74058.
[85] Weber, A. I. and Pillow, J. W. (2017). Capturing the dynamical repertoire of single neurons with generalized linear models. Neural computation, 29(12):3260-3289.
[86] Williams, A. H., Poole, B., Maheswaranathan, N., Dhawale, A. K., Fisher, T., Wilson, C. D., Brann, D. H., Trautmann, E. M., Ryu, S., Shusterman, R., et al. (2020). Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping. Neuron, 105(2):246-259.
[87] Williams, C. K. and Rasmussen, C. E. (2006). Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA.
[88] Wilson, J., Borovitskiy, V., Terenin, A., Mostowsky, P., and Deisenroth, M. (2020). Efficiently sampling functions from gaussian process posteriors. In International Conference on Machine Learning, pages 10292-10302. PMLR.
[89] Wilson, J. T., Borovitskiy, V., Terenin, A., Mostowsky, P., and Deisenroth, M. P. (2021). Pathwise conditioning of gaussian processes. The Journal of Machine Learning Research, 22(1):4741-4787.
[90] Xiao, S., Yan, J., Yang, X., Zha, H., and Chu, S. (2017). Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31.
[91] Yang, R., Gupta, G., and Bogdan, P. (2019). Data-driven perception of neuron point process with unknown unknowns. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, pages 259–269.
[92] Yousefi, A., Amidi, Y., Nazari, B., and Eden, U. T. (2020). Assessing goodness-of-fit in marked point process models of neural population coding via time and rate rescaling. *Neural Computation*, 32(11):2145-2186.
[93] Yu, B. M., Cunningham, J. P., Santhanam, G., Ryu, S., Shenoy, K. V., and Sahani, M. (2008). Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. Advances in neural information processing systems, 21:1881-1888.