Three-dimensional spike localization and improved motion correction for Neuropixels recordings
Authors: Julien Boussard
Erdem Varol
Hyun Dong Lee
Nishchal Dethe
Liam Paninski
Venue: NeurIPS 2021 Poster (2021)
Keywords: Spike-sorting
Neuropixels
Localization
Registration

Abstract: Neuropixels (NP) probes are dense linear multi-electrode arrays that have rapidly become essential tools for studying the electrophysiology of large neural populations.  Unfortunately, a number of challenges remain in analyzing the large datasets output by these probes.  
Here we introduce several new methods for extracting useful spiking information from NP probes.  First, we use a simple point neuron model, together with a neural-network denoiser, to efficiently map spikes detected on the probe into three-dimensional localizations.  Previous methods localized spikes in two dimensions only; we show that the new localization approach is significantly more robust and provides an improved feature set for clustering spikes according to neural identity (``spike sorting").  Next, we apply a Poisson denoising method to the resulting three-dimensional point-cloud representation of the data, and show that the resulting 3D images can be accurately registered over time, leading to improved tracking of time-varying neural activity over the probe, and in turn, crisper estimates of neural clusters over time. The code to reproduce our results and an example neuropixels dataset is provided in the supplementary material.

Three-dimensional spike localization and improved motion correction for Neuropixels recordings
Julien Boussard'
Erdem Varol"
Hyun Dong Lee
Nishchal Dethe
Liam Paninski
Department of Statistics and Neuroscience, Center for Theoretical Neuroscience, Grossman Center for the Statistics of Mind, Zuckerman Institute, Columbia University; * *-equal contribution
Abstract
Neuropixels (NP) probes are dense linear multi-electrode arrays that have rapidly become essential tools for studying the electrophysiology of large neural popula- tions. Unfortunately, number of challenges remain in analyzing the large datasets output by these probes. Here we introduce several new methods for extracting use- ful spiking information from NP probes. First, we use simple point neuron model, together with a neural-network denoiser, to efficiently map single spikes detected on the probe into dicensiimeiional localizations. Previous methods localized indi- vidual spikes in two dimensions only; we show that the new localization approach is significantly more robust and provides an improved feature set for clustering spikes according to neural identity ("spike sorting"), ). Next, we denoise the resulting diree-dimensional point-cloud representation of the data, and show that the result- ing 3D images can be accurately registered over time, leading to improved tracking of time-varying neural activity over the probe, and turn, crisper estimates of neural clusters over time. Open source code is available at https: //ttps://giii //github thub commintt tum/int-un www.-baannnnnaaiaaaaeeeaeeeeeaee
babkssikkes... http://ppees__ocllileeeeeeeeee localloationnnn reggisstaii... git
Introduction
Neuropixels (NP) probes are dense linear multi-clectrode arrays that enable the simultancous obser- vation of hundreds of neurons across multiple brain areas. Since their introduction in [13 (see also (27)), they have rapidly become an essential neurotechnology. deployed in hundreds of labs around the world. A number of challenges remain in analyzing the large datasets output by these probes. The basic goal "spike sorting" i.e., detect action potentials ("spikes") and assign these spikes to individual neurons. Despite significant effort in the field 4 12 27], current spike sorters for NP data have trouble tracking temporally non-stationary data and accurately sorting small spikes (28].
One major advantage NP probes their spatial density: the high spatial resolution of electrodes on the probe implies that each extracellular spike will typically be detected at multiple sites, providing an opportunity to "triangulate" the location of each spike. If effective, this spike localization can be helpful for multiple downstream tasks: spike locations serve useful low-dimensional summariza- tions of the high-dimensional spatiotemporal spiking signals, which can in turn be visualized and used as features for clustering and tracking as the probe undergoes small motion relative to the brain.
Several spike sorting algorithms localize spikes using a simple center of mass (CoM) method. More precisely, for each waveform, they select a set of channels, and compute the weighted average of the selected channels' positions, where the weights are given by an estimate of the amplitude of the spike on each channel. This fast, cheap method gives informative location estimates for spikes in multi-electrode array (MEA) recordings where long dendritic or axonal signals are not prevalent
(23 120 [0] (cf. the primate retina [17], where long axonal signals make localization approaches less useful). However, by definition, the CoM approach localizes spikes inside the convex hull of the electrodes, which is especially problematic for Neuropixels recordings, due the long and thin shape
of the electrodes. Hurwitz et al. III propose an improved localization method that provides location
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

estimates that can lie outside of the probe, but the estimates are still (like the CoM method) restricted the two-dimensional plane spanned by the electrode locations. In addition, several approaches have been developed for localizing templates (averages over many spikes, post spike-sorting) in three
dimensions, using biophysical models of varying sophistication [26] 25 5]. Finally, triangulation approaches based on point-source [6] 15] or dipole models [19] have been proposed for smaller-scale tetrode recordings, in which spikes are only detected on four electrodes, limiting the resolution the resulting localization in the case of small, noisy spiking units.
In this paper, we build on this previous work, making four key contributions:
1. We introduce a denoise mennois--teen-rranee then-ttrinngllate approach, based on simple point neuron model, that localizes single spikes in three dimensions.
2. This improved three-dimensional localization in turn enables better spatial clustering of distinct neural units characterized by unique action potential waveform shapes.
3. Our localization further enables better motion estimation in recordings with drift.
4. Our approach is robust to different probe geometries. We demonstrate its utility and performance in both Neuropixels 1.0 and 2.0 probes.
Neuropbesls probe
Point neuron model
NN Dancising and Collaiin Remooal
Intering locations
(i,
Vavafomm
M
C
Naurozieats 1.0 Loca Izations
Neurapixels 2,0 Locakzationss
CoM Xirm
Hurwitz
mmm
Ours
X0AmI
Figure 1: Overview of the proposed localization technique. (A) The peak-to-pes to-peak voltage ampli- tude attenuation recorded at each channel of the Neuropixels probe is modelled as approximately inversely proportional the distance to a spiking unit point source. The 3D spatial location of the spiking unit is then inferred by essentially solving triangulation problem (minimizing least-squares loss all the channels that detected the spiking event). (B) Since the triangulation depends on the peak-to-peak, amplitudes of spiking events, we denoise the amplitudes using neural net and further disambiguate distinct units that have spiked at the same time (collision removal). Each row shows single spike event (original and denoised), captured on multiple nearby electrodes. (C) We apply our method to spatially map the localizations of over one million spiking events in both Neuropixels 1.0 and Neuropixels 2.0 recordings. Our localization is compared with the oftteoo-thee-ase art local- ization method proposed in [11], as well as the baseline method of inferring position using center of mass (CoM) of channel positions weighted by the recorded amplitude. Spikes colored by maximum amplitude, normalized by the recording baseline noise level ("standardized units' 's.u."). Yellow corresponds to high and dark blue low amplitude.
(D) Our localization yields groups of spiking events that disperse 3D space and cluster based on amplitude. Colors correspond to clusters of waveforms found by YASS (17), for visualization purposes only. Orange squares represent http:///ww.....ooiiiiuueeeeeeeee
recording electrode positions. Datasets from 2020hhlo//materdoo//otteeaaa md 277; see video--igure-I.mpp for Datoviz (24) visualization.

---

2 Methods
2.1 Point neuron model
A neuron's 's spike is the result of a quick change in its membrane potential due to the shift of its charged particles from one side of the membrane the other. This movement of charge creates an electric field outside of the neuron [14], and Neuropixels electrodes record the potential of this field, multiple locations along the probe (Fig. TA]. We use simple point-source model for these voltage differences at distance from the cell:
V
where a represents the cell's overall signal magnitude, and the parameter b is fixed independently of the cell. This point-source model has been previously applied in tetrode recordings [ [.] and could be replaced with more detailed models (dipole, ball-and-stick, etc) if desired.
2.2 Spike denoising and collision suppression
The triangulation approach described below relies only on the amplitude of the spike on each electrode. In practice we measure this amplitude using the peak-to-peal (PTP) value (maximum minus minimum voltage over a short temporal interval). In the presence noise, the PTP biased upwards (since noise will increase the maximum and decrease the minimum signal), and this bias depends on the underlying signal strength (the PTP of small spikes more biased than large spikes). Furthermore, collisions from near-synchronous spike events are prevalent in dense multi-electrode data (17), and even small collisions can lead to inaccurate location estimates if not accounted for properly.
Therefore, it is critical to incorporate a denoising collision- suppression step prior to triangulation. We use the neural net denoiser implemented in YASS [7] advantage of the waveform's 's spatiotemporal signature, to suppress both noise and collisions (Note that simple image denoising methods such as 16 3 do not serve suppress collisions, and are therefore less effective here.). Fig. 1B shows three examples of waveforms (in blue) and their denoised version (in red). The neural net denoiser removes noise and collided spikes, allowing us to successfully estimate each spike's PTP amplitude; see the Appendix for further details.
(retrained on NP data), which takes
2.3 Spike localization
Given the denoised PTP amplitudes obtained above, we can proceed with localizing the origin of each waveform with respect to the multiple electrodes on the NP probe. We will denote {x, } the coordi- nates of the point neuron, and {xe, = the planar coordinates each NP electrode, with ye set to zero by convention. Here we exploit the fact that each spike is detectable on multiple channels simultane- ously: i.e., we have multiple observations of the form ptp. Vi2+12 V(e ((e-ne)²+++ +2y2
and our goal is to infer y, z, and a. Thus, for each spike and selected set of channels C around the "main" channel for each spike (i.e., the channel with the largest PTP amplitude), we want to optimize:
Σ aEC
ptp.
xo)
See Appendix for full optimization details. (One important note here: due to the planar NP electrode geometry, the sign of y is not recoverable, i.e., we can't infer whether spike in front of or behind the probe, even if we can infer the orthogonal distance from the probe plane.) Fig. (C) shows examples inferred [x,z) locations of detected spikes in both NP1.0 and 2.0 probes (which have different electrode layouts), colored by max PTP. Panel (D) shows example of 3-d [x, y, 2) localization for the NP2.0 data.
2.4 Point cloud image denoising
The estimated localizations are a point cloud in continuous space (Fig. A To enable the use of image registration techniques, we convert the point cloud representation an image representation by generating spatial histograms where pixel location and intensity capture the spatial coordinates of

---

A
10
20
Localization point set (continuous)
30
B
10
Xum)
20
Localization image (discretized)
X
30
C
10
20
Poisson denoised localization image
30
50
100 z(um)
150
200
Figure 2: Poisson denoising point sets of localizations yields interpretable images of spiking unit locations. (A) The estimated spiking unit localizations a sparse point set continuous space. (B) By binning spike locations in a spatial histogram, we genérate a localization "image" from the point set input, where pixel intensities that denote the average amplitude in positions in space roughly follow Poisson distribution. Note that this image is also sparse. (C) Modelling the noise distribution of this image as Poissonian enables the use of Poisson image denoising techniques [18]. The resulting de- sparsifiied pixelwise representation of spiking unit locations is more amenable to downstream tasks such as image registration and visualization (see Fig.
spikes and their mean amplitudes (Fig. B), following [27]. Note that this image representation is sparse and resembles a speckled Poisson image [88. due to the limited number of spikes that are captured in each spatial position. In order to compute more accurate motion estimates, it is crucial to appropriately denoise this image representation. We adopt Poisson denoising technique [8]. with three steps. First, we apply the Anscombe root transformation [ to the histogram. Second, we use a Gaussian denoiser (e.g. Block-matching and 3D filtering 77. or non-local means [3]) on the transformed histogram image. Third, the denoised signal is obtained by applying inverse transformation to the denoised transformed histogram. Fig. 2 demonstrates the output for one second of time-binned data: Poisson denoising yields more continuous pixelwise representation of the spatial histograms, which satisfies the pisel-wise continuity assumptions image registration algorithms [9] and enables more robust estimate of displacement.
2.5 Motion inference and registration
Given of dense Poisson denoised localization images for each second of recording, esti- mate relative motion between frames by treating the set of images as a time-series video and use image registration techniques. Several techniques have been recently introduced to infer motion in Neuropixels recordings. The method proposed in 27 follows a non-rigid template based approach, where average image of localizations used using a phase-correlation based displacement estimator 9 In contrast, the method (29 follows decentralized non-rigid approach, treating each localization image as its own template performing decentralized pair-wise motion estimates. This method has been shown to be more robust than the template based approach and utilize it here to infer displacement both within the NP plane (z, x) and orthogonal to this plane (y) for the localization images. For other methods that yield only planar localizations, we just estimate vertical (z) and horizontal displacement (y). The motion estimates for un- un-denoised and Poisson denoised localization image sets are shown in Fig. 5D. Once we estimate motion estimates for each time frame, we then correct for motion by volumetrically (or planarly) translating each image by the negative of the local non-rigid displacement that is estimated for that frame that location on the probe, bringing all frames into alignment. Example average images before and after alignment can be seen in Fig. H.
anchor register individual localization images

---

2.6 Point-cloud registration
In addition to the image-based registration approach described above, we also experimented with direct point-cloud registration approaches, which skip the formation of intermediate image repre- sentation and are therefore amenable to applications involving higher-dimensionn Representing the spikes in each second of data point set, remove outliers [17] and compress the point cloud via simple hierarchical clustering, parameterized by a single maximum merge distance variable. Then, for each pair of compressed point clouds, we run the iterative closest point (ICP) algorithm (2) to find the displacement in the 2-direction that minimizes the distance between the clouds, weighted by amplitude and number of spikes. To avoid poor local optima, we utilize coarse grid search in to initialize the algorithm; see the supplement for full details. The two motion inference methods lead to comparable displacement estimates the datasets presented in this paper.
spike fcaturizations.
3 Results
3.1
Comparing localizations and clustering
Fig. 3 's first three panels show the (x, z} inferred location of detected spikes in 200 seconds of a Neuropixels 2.0 recording, using center of mass (CoM), the Hurwitz et al. [ method and the new proposed method. (For the Hurwitz al. method, channels within 35 jm from the main channel are included and amplitude jitter is set to 0 vV.) While CoM does not allow us to separate waveforms along the x-axis or localize outside the convex hull of the electrode, Hurwitz et al. and our method induce spike clusters that appear very different. Hurwitz et al.'s clusters tend to be more isotropic and often appear "paired" on both sides the probe, while our method leads to differently sized clusters, without pairing. Low amplitude clusters are more spread out than higher amplitude clusters, which is expected as noisier amplitude measurements will lead to noisier localizations.
Our method infers two additional features, and a. Fig. 3 's two rightmost columns show scatter plots of z vs y and a. From the clusters in boxes 2.3,4 in the right column, it appears that a can separate waveform clusters more strongly than y, suggesting that this variable contains useful information about the spatial shape of each spike; see the Appendix for further exploration.
The four boxes in blue, green, pink, and red highlight the different spike locations corresponding to similar z-location for each method. For the spikes inside these boxes, we clustered Hurwitz al. {x, z} locations, and our (x, z,0) features using Gaussian Mixture Model with 2 or 3 components. The corresponding waveforms are shown in Fig. 33.. The new method provides visually improved waveform clustering here; the separation induced by is also coherent with the waveform shapes.
A striking difference between the Fig. 3 left and right columns' waveforms is that the right column waveforms appear much less noisy. Most of Hurwitz et al. waveforms appear not centered, but really collided waveforms representing distinct neural units that poorly disambiguated. In panels 1, 2, and 3, the Hurwitz et al. method localizes small amplitude spikes collided with large amplitude ones in the same region the largest amplitude spike. This highlights the importance of using the individull-wveefrr
neural network denoiser. In the supplementary material we contrast the results of all three localization methods using denoised raw waveforms. Finally, we find that the Hurwitz method sometimes experiences instabilities in the localization leading to separated clusters in the estimated localization space that do not correspond to separations in the raw data. Figure] 3 panel 4 illustrates this idea, as many spikes of similar amplitude are located by Hurwitz et al. method on either the right left side of the probe (yellow or red clusters).
3.2 Visualizing spike densities after motion correction and registration
After localization and image-based denoising and alignment (as described in the Methods), we compute the average post-aligned image to visualize the volumetric spatial layout of distinct spiking units in the vicinity of the probe. The particular recording that we analyze has been inserted through the cortex, hippocampus, and thalamus of a mouse brain (see Fig. 4A for an approximate insertion positioning). The average localization image before and after motion correction and registration for the full probe length is shown in Fig. 4B. where we can make out three spatially distinct populations of neurons. Using the depth information and the approximate insertion order, we can then identify and annotate the specific anatomical regions that the neurons reside in and zoom in to observe
5

---

A
CoM
Hurvizz
Oura
Alpha
802
2 (um)
500
(um)
(um)
(umn)
Alpha (normalized)
Clustering based Hurwitz's localization
Clustering based localization alpha feature
Angulitudo ($.u) Amplitudo (S.0.
-30
:00
Amallibaoo uu Amplltudo (5.U3
-30
Channels
Channels
Figure 3: Inferred 3D spatial features yield improvements in waveform clustering. (A) {x, 2) locations of spikes detected from 200 seconds of NP 2.0 recording, inferred by the center of mass baseline (left), Hurwitz et al.'s [[]] state-of-the-an method (second to left), and our method (middle). The scatter plots on the right of the figure represent vs. y and a, where y and a are additional features our method learns. The four colored boxes frame spikes that are shown in panel B, and the yellow, blue and red color of the spikes represent GMM cluster assignments, using (x, z) and (x, z, a as features for Hurwitz method and our method, respectively. The number of clusters has been chosen by hand to reflect the shape of the point clouds in each box. Orange squares represent the position of NP 2.0 recording channels. The boxes in each column are not the same size, trying to match spikes localized in the same z-area, which depends on the localization method. (B) Waveforms corresponding to each box and cluster, for both Hurwitz et al.'s (left) and our method (right). (We don't attempt to cluster based on CoM features, due to the lower resolution of the CoM output.) Note that there is clear one-to-one correspondence between the left and right waveforms, as our denoising collision removal can change localization drastically. The many non-centered waveforms in the left column show collided spikes that have not been localized properly Hurwitz et al. method lacks collision removal. In addition, we find that the Hurwitz et al. approach often spatially separates similar waveforms, while failing to isolate different units. On the other hand, our location- based clusters correspond to visibly apparently separate units, without corruption from poorly localized collided spikes. Moreover, the additional feature waveforms when (x, z) features alone cannot (red box). See supplementary material for further details.
correctly separate different

---

Probe insertion
Average one-second time bin poisson image (max-y-projection) -Full probe length
A
Cortex
1966
alamnj
Hippocampus
Thalamus
Average one- nec-secodd time bin poisson image (max-yyprojection) - Zoomed regions
Max-z-prroycttoo
Unregistared
100
100
Registered
ybmm
Unnegielered
Cortex
300
stum)
20
86
Uneejiteredi
Unoogstered
100
Plgg Storao
120
Registered
Rogistered
x(um)
300
Hippocampus
Thalamus
Figure 4: Averaging motion corrected and registered localization images enables a crisp visu- alization of spiking density. (A) We analyze an NP 2.0 recording that has been inserted in the regions of cortex, hippocampus, and thalamus of mouse brain. The average of thousand one-second localization images after motion correction and registration yields crisp mapping of neural units in three distinet anatomical regions. Max y-dimension visualization is shown in panels (B,C) and max-2-dimension visualization is shown panel (D). (C) Zooming into cortical, hippocampal, and thalamic depth regions shows significant improvement of the crispness of the average localization image after registration. Note that motion effects appear as long blurry streaks in averaged images, whereas well corrected motion yields globular and bright individual clusters. (D) Maximum pro- jection along the z-dimension emphasizes localization quality as a function of depth and shows a crisper localization of units near the probe and more uncertain localizations farther away (in depth), owing to higher localization uncertainty for more distant, smaller spikes; this effect is also visible in Fig. 3A. See supplementary material and video video-figure-4.mpp for a detailed comparison of unregistereddregisteeed frames.
their densities Fig. 4B,C. Note that there is a higher density of spiking units in the thalamus (Fig. (C-bottom-right) By contrasting the unregistered average images with the registered ones, we see the effects of motion, such blurred streaks of cell shapes, mitigated after registration, yielding sharper images of neural populations. Also, visualizing the average image projected along the length of the probe (z-axis) shows that while registration does improve sharpness of localization in units close to the probe, "deeper" units that are further away from the probe (and therefore have lower amplitude) remain localized with more uncertainty, visible also in Fig. 3A.
3.3 Motion inference and registration evaluation
To further quantify whether our localization and Poisson denoising enable better motion estimation and image registration, we compare the resulting motion estimates and registration metrics using the localization images generated with our method versus the localizations from Hurwitz al. [I]] and the center-of-mas localizer (CoM). To evaluate registration quality, we utilize a Neuropixels 2.0 recording with physically introduced exaggerated motion using an actuator motor (Fig. ABB) The details of this publicly available recording are shared in (77) The motion estimates based on all three localization methods and using raw versus denoised localization images can be seen Fig. 5D. The main conclusion is that Poisson denoising significantly improves the motion estimation jitter for all methods compared to utilizing the raw localization images. Furthermore, our localization method affords the estimation of an additional dimension of motion, namely along the depth dimension (y-axis) to yield more refined motion estimates. To evaluate motion estimation quality along the length axis (z-axis), we provide raster plots before and after registration using all three localization

---

Actuator motion
B
Recording
Poisson image denoising / Image generation
Amplitude (S.U.)
Ours
Channels
Max-y
100
A
x(arr) Max-7
Hurwitz
Planar localization (2D)
Samples (30kHzz)
D
Motion estimates
Z-motion
X-motion
Y-motion
CoM
Planar localization (2D)
Ours
Depth based raster plots (unregistered/rreiisteeed
Ours
Hunwitz
CoM
Displacement (um)
www
Hurwitz
zomi
N/A
COM
2000
1000 CN
2000
1000
2000
N/A
Correlation to average frame
RMSE
1000
denoiied
Time bins (s)
0.11
Noot nooi-denciedd
Ent. (bits) Unvegistered;
Registered
Difference
Ours
1.0713 3 0,9282
0,1431
200
400
600
600
1000
Hurwitz
1,2645
..111.9.994..
1,1994
0,0651
Unragistered
Registered
CoM
1.5412
1.4776
0.0636
G
Time bins (s)
Ours
Average 1000 bins unregistered/registeeeed images
Hurwitz
Unragistered
Unragistered
Registered
Jnregisitree
CoM
Registered
Recistered
Figure 5: Improved localization enables better motion correction and registration of Neuropix- els recordings. (A, B) We analyze mouse NP 2.0 recording that has been subjected to mechanical saw-tooth motion [27]. (C) The 3D localization images spikes after Poisson denoising sand second recording are displayed in comparison to the planar localizations provided by Hurwitz et [IT] and the center mass technique. Poisson denoising the localization point clouds for all three techniques yields continuous image representation of spiking unit locations and frame video representation (one frame for each second of data). (D) We apply the existing registration technique [29] time-binned image representations of data to estimate the amount of z, and y motion for all three localization techniques. We show the motion estimate for each localization technique, with and without Poisson denoising. Poisson denoising significantly improves the noise jitter motion estimation. (E) Visualizing z-direction raster plots of the unregistered and registered recordings (after Poisson denoising) shows stabilization of motion effects for all three methods with nominal improvements by our method over others. Green arrows denote areas of the raster plot that have been well stabilized using localization versus the localization of Hurwitz al. (F, H, H, K) Visualizing the average image after registration using our localization shows significant decrease in image entropy (as measure of localization "sharpness") over compared methods. (G) Additionally, our localization affords the highest average correlation of registered images to the average image and the lowest RMSE. Note that CoM method's high average correlation after registration should be contrasted with its high values prior to registration. Since this localization provides highly blurred images, the average correlation after registration is vacuously high. See supplementary material and video (video-figel fioofffuur--S.mpp))). for more detailed comparisons of unregistered/registeeed frames using the three localization techniques.
thou-
thousand

---

methods after Poisson denoising (Fig. 5E.. This is essentially a maximum y- and x- projection of registered images, retaining a singleton dimension z- evaluate residual jitter. The registered rasters using localization images from all three methods are qualitatively well stabilized, with nominal improvements afforded by our localization (highlighted by green arrows in (Fig. EE.. Evaluating the registration performance using all dimensions shows that our localization, thanks to its three- dimensional spread of features, provides better reduction of residual motion (Fig, (5,G) and sharper alignment of images (Fig. (JJ.K) compared to the registration using Hurwitz et al. [II] and CoM localization images.
In further detail, we evaluate registration performance using three metrics, First, we compute the average correlation of aligned images the average image as commonly evaluated in the image registration literature [22]. We contrast this with average correlation before alignment to note the improvement in correlation that alignment provides. Note that the CoM average correlation is high both before and after alignment, denoting that this score is not due to good motion correction but rather to the overall spread of CoM localization that vacuously yields a high correlation to the average image regardless of motion correction. In this metric, our method outperforms both Hurwitz et al. and CoM. We also compute the RMSE the individual frames to the average frame after registration. Our method also yields the lowest RMSE, showing that individual frames align well to the average frame. Lastly, we evaluate the "sharpness" of the average frame by computing its image entropy. In this regard, our average frame is shown to be qualitatively (Fig. H)J,K) and quantitatively (Fig. 5F) sharper than the results from the compared methods.
4 Conclusion
In summary. we provide a simple denoising and point-model triangulation approach to infer three- dimensional source locations from individual spikes in Neuropixels recordings [33 [27.. This localization shown to facilitate better clustering of distinct neural units as well as enabling better estimation and correction of motion in datasets where the spiking units experience non- non-stationarities due to probe drift. We also demonstrate the benefits of converting the spiking unit localization point sets into continuous images using Poissonian denoising [18), enabling informative visualizations of the neural source distributions in the vicinity of the Neuropixels probe.
Two open directions are clear for future work. First, as emphasized above, we have used a highly simplified point model perform localization here. Elaborations of this model should lead to improved accuracy, though robustness and speed of the resulting localizer are also critical for downstream applications and should be balanced accordingly. Second, establishing the accuracy of the proposed methods experimentally will be a challenging but critical next step.
Broader impact
Neuropixels probes are a recently- deveell-de neurotechnology that enable us to record from hundreds of neurons simultancously in multiple regions of the brain. Our work will improve scientific conclusions derived from Neuropixels recordings; downstream, we expect related methods to also improve the performance of brain machine interfaces based extracellular neural recordings.
Acknowledgments
We thank Nick Steinmetz for collecting and sharing Neuropixels recordings, Olivier Winter and the International Brain Lab for testing our methods, Cole Hurwitz and Matthias Henning for helping us draw comparisons with their localization method, as well as Alessio Buccino and Samuel Garcia for helpful discussions and references. We also thank Cyrille Rossant for providing support for the Datoviz platform. This work was supported by the following grants: Gatsby Charitable Foundation GAT3708, NSF DBI-1707396 NSF 1546296, NIH U19N$104649, NIH U19NS123716, Simons Foundation 543023, Wellcome Trust 209558, and Wellcome Trust 216324.

---

References
[1] F. J. Anscombe. The transformation of poisson, binomial and negative- binomial data. Biometrika, 35(3/4) 254, 1948.
[2] J. Besl and N. D. McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pages 1866606. International Society for Optics and Photonics, 1992.
[3] A. Buades, B. Coll, and J.-M. Morel. Non-local means denoising, Image Processing On Line, 1:208-211 212, 2011.
[4] A. P. Buccino, C. L. Hurwitz, S. Garcia, J. Magland, H. Siegle, R. Hurwitz, and M. H. Hennig. Spikeinterface, unified framework for spike sorting. eLife, 9re61834, nov 2020.
[5] A. P. Buccino, M. Kordovan, T. V. Ness, B. Merkt, P. D. Häffiger, M. Fyhn, G. Cauwen- berghs, S. Rotter, and G. T. Einevoll. Combining biophysical modeling and deep learning for multiclectrode array neuron localization and classification. Journal of Neurophysiology, 120(3):1212-12222 2018. PMID: 29847231.
[6] M. I. Chelaru and M. S. Jog. Spike source localization with tetrodes. Journal of Neuroscience Methods, 142(2):305-315, 2005.
[7] K. Dabov, A. Foi, V. Katkovnik, and Egiazarian. Image denoising with block-matching and 3d filtering. In Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, volume 6064, page 606414, International Society for Optics and Photonics, 2006.
[8] I. Delgado Ruz and S. Schultz. Localising and classifying neurons from high density mea recordings. Neurosci Methods., 233:115-122 2014.
[9] M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup. Efficient subpixel image registration algorithms. Optics letters, 33(2):156-1555 2008.
[10] G. Hilgen, M. Sorbaro, S. Pirmoradian, J.-O. Muthmann, I. E. Kepiro, S. Ullo, C. J. Ramirez, A. Puente Encinas, A. Maccione, L. Berdondini, V. Murino, D. Sona, F. Cella Zanacchi, E. Sernagor, and M. H. Hennig. Unsupervised spike sorting for large-scale, high-density multielectrode arrays. Cell Reports, 18(10):25112552. 2017.
[11] C. Hurwitz, K. Xu, A. Srivastava, A. Buccino, and M. H. Hennig. Scalable spike source localization in extracellular recordings using amortized variational inference. arXiv preprint arXiv: /237, 2019.
[12] J. Jun, J. Magland, C. Mitelut, and A. Barnett. Ironclust: Scalable and drift-resistant spike sorting for long-duration high-channel count recordings; 2020.
[13] J. J. Jun, N. A. Steinmetz, J. H. Siegle, D. J. Denman, M. Bauza, B. Barbarits, A. K. Lee, C. A. Anastassiou, A. Andrei, Ç. Aydin, et al. Fully integrated silicon probes for high-density recording of neural activity. Nature, 551 (7679):232-22) 236, 2017.
[14] C. Koch. Biophysics of Computation: Information Processing in Single Neurons (Computational Neuroscience Series). Oxford University Press, Inc., USA, 2004.
[15] T. Kubo, N. Katayama, A. Karashima, and M. Nakao. The 3d position estimation of neurons in the hippocampus based on the multi-site multi-unit recordings with silicon tetrodes. In 2008 30th Annual International Conference of the IEFE Engineering in Medicine and Biology Society, pages 5021-5024, 2008
[16] J. Lecoq. M. Oliver, J. H. Siegle. N. Orlova, and C. Koch. Removing independent noise in systems neuroscience data using deepinterpolation. bioRxiv, 2020.
[17] J. Lee, C. Mitelut, H. Shokri, I. Kinsella, N. Dethe, S. Wu, K. Li, F. B. Reyes, D. Turcu, E. Batty, al. Yass: Yet another spike sorter applied to large-scale multi eleti-electrode array recordings in primate retina. bioRxiv, 2020.
10

---

[18] M. Makitalo and A. Foi. Optimal inversion of the anscombe transformation in low-count poisson image denoising. IEEE transactions on Image Processing. 20(1):99-109, 201,
[19] Mechler, J. D. Victor, I. Ohiorhenuan, A. M. Schmid, and Q. Hu. Three-dimensional localization of neurons in cortical tetrode recordings. Journal of Neurophysiology, 106(2):828- 848, 2011. PMID: 21613581.
[20] J.-O. Muthmann, H. Amin, E. Sernagor, A. Maccione, D. Panas, L. Berdondini, U. S. Bhalla, and M. H. Hennig. Spike detection for large neural populations using high density multielectrode arrays. Frontiers in Neuroinformatics, 9:28, 2015.
[21] M. Pachitariu, N. Steinmetz, S. Kadir, M. Carandini, et al. Kilosort: realtime spike-sorting for extracellular electrophysiology with hundreds of channels. BioRxiv, page 06 061481, 2016.
[22] E. Pnevmatikakis and Giovannueci. Normcorre: An online algorithm for piecewise rigid motion correction of calcium imaging data. Journal of neuroscience methods, 291:83-94 2017.
[23] J. S. Prentice, J. Homann, K. D. Simmons, G. Tkačik, V. Balasubramanian, and P. C. Nelson. Fast, scalable, bayesian spike identification for multi-electrode arrays. PloS one, 6(7), 2011.
[24] C. Rossant, N. P. Rougier, et al. High-performant interactive scientific visualization with datoviz via the vulkan low-level gpu api. Authorea Preprints, 2021.
[25] Z. Somogyvári, D. Cserpán, I. Ulbert, and P. Érdi. Localization of single-cell current sources based on extracellular potential patterns: the spike esd method. Eur J Neurosci, 36(10), 2012.
[26] Z. Somogyvári, L. Zalányi, I. Ulbert, and P. Érdi. Model-based source localization of extracel- lular action potentials. Journal of Neuroscience Methods, (47(2):126-13))) 2005.
[27] N. A. Steinmetz, C. Aydin, A. Lebedeva, M. Okun, M. Pachitariu, M. Bauza, M. Beau, J. Bhagat, C. Böhm, M. Broux, S. Chen, J. Colonell, R. J. Gardner, B. Karsh, F. Kloosterman, D. Kostadi-
nov, C. Mora-Lopez. O' CaClllghaan, Park, J. Putzeys, B. Sauerbrei, R. J. J. van Daal, A. Vollan, S. Wang, M. Welkenhuysen. Z. Ye, J. T. Dudman, B. Dutta, A. W. Hantman, K. D. Harris, A. K. Lee, E. I. Moser, J. O'Keef, A. Renart, Svoboda, M. Häusser, S. Haesler, M. Carandini, and T. D. Harris. Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings. Science, 372(6539), 2021.
[28] N. A. Steinmetz, C. Koch, K. D. Harris, and M. Carandini. Challenges and opportunities for large-scale electrophysiology with neuropixels probes. Current opinion in neurobiology, 50:92-100, 2018
[29] E. Varol, J. Boussard, N. Dethe, O. Winter, A. Urai, The International Brain Laboratory. A. Churchland, N. Steinmetz, and L. Paninski. Decentralized motion inference and registration of neuropixel data. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1085-1089, IEEE, 2021.


Reviews:
[{"review_id": "rbCJHmJCwch", "reviewer": "Reviewer_PJEK", "summary": "The authors develop a new analysis pipeline for the detection and localization of spikes in Neuropixel recordings. The pipeline is based on a denoising step applied to all channels, detection of spikes via their amplitudes, triangulation across channels to localize spikes, and finally a motion correction step to counter potential movements of the electrode. The efficiency of the method is demonstrated by comparing results against several state-of-the art methods.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well written and everything seems technically correct. The results are nicely demonstrated on various data sets, including those with (partial) ground truth data (motion). I think this paper will be of great interest to anyone recording neural activities with Neuropixel probes. \n\nThe only potential downside I see is that the proposed analysis pipeline consists largely of a sequence of analysis methods taken from the literature. While several of these methods are applied for the first time to the problem of spike localization (such as triangulation), from a purely theoretical or algorithmic point of view, the paper provides little (new) insights. In other words, the paper may not be the best fit for the Neurips audience.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Has been addressed.", "review_text": "", "extra_scores": {"ethical_concerns": "No concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2h"}, "extra_text": {}}, {"review_id": "njHSkKiQzj", "reviewer": "Reviewer_vagG", "summary": "This paper proposes a novel spike sorting algorithm, specifically for dense arrays such as the NeuroPixels series of electrodes. This is an extensive and detailed report. The comparisons to other approaches are up-to-date and thorough. They present some nice analysis of electrode drift, showing that the new algorithm performs well. It is well written, clear, and includes extensive figures to illustrate the work and results. This is an excellent piece of work.", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "They use a point-source model for neurons, then denoise and supress collisions, and finally do triangulation in 3D. This apparently improves clustering fidelity. They also track electrode drift and their algorithm offers excellent performance for that aspect as well.\n\nThis work can have impact on both fundamental science, as well as BCI work-- although the latter often disregards spike sorting.\n\nIt would be helpful if the report could show some spike raster data comparing their approach and that of Hurwitz and CoM. For example, in Fig 2 there are some clusters that were split in their algorithm and not with Hurwitz et al.'s algorithm. What do those spike rasters look like? Sometimes good spike sorting is evident when aligned to a stimulus or task parameter, e.g., Fig 3 in https://onlinelibrary.wiley.com/doi/epdf/10.1111/joa.12228\n\nThe main limitation is that there is no ground truth data, or calibration data. How can we say that this algorithm is better when we don't know what the truth is? This is a general problem for the field, and not a specific problem for this paper, but it does make it awkward to strongly advocate for a particular spike sorting method. Still, I am enthusiastic about this work and strongly support it.\n\n\"crisper estimates of neural clusters over time\" - This is an unclear, qualitative way to say it. Please use more precise, quantitative language. Perhaps: \"more stable unit localization over time, as measured by image entropy\"? I defer to the authors, and I will not insist on this, but I would like to encourse the use of more precise, quantitive language.\n\nThe model (a point neuron model) is rather simple, which is maybe good, but where does it break down? Spike signals can emerge from different compartments of a neuron. Spike amplitude can be affected by not only distance, but also geometry (e.g., angle of an axon relative to the electrode) and intervening tissue (e.g., glia in front of an electrode). I guess those would just get folded in, and not cause problems as long as they are stable in time-- degrading the precision of the localization, but still aiding the clustering/sorting. I would appreciate the authors commenting on that.\n\nPurely curiosity: The 3-step Poisson denoising is interesting, and seems to work well. I wonder what the author(s) think about the DeepInterpolation work. https://github.com/AllenInstitute/deepinterpolation Have they tried to replace their denoising with this approach and examined performance?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No concerns.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "fK9pIkBvr0x", "reviewer": "Reviewer_2yf2", "summary": "In “Three-dimensional spike localization and improved motion correction for Neuropixels recordings”, the authors describe a pipeline for denoising and localizing spikes from dense extracellular recordings of neurons. Specifically, recordings are denoised using a previously published neural network based technique, then 3D locations are inferred using a straightforward point-source model. Finally, spikes are localized using Poisson image denoising.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The manuscript is well-written, and I found the methods easy to follow and design decision are reasonable and justified. This paper in its current form is well-above the acceptance threshold for NeurIPS, and I only have a few comments:\n\nMajor:\n\n1)\tIn the comparison with Hurwitz’s method, my understanding is that the authors do not use any spike amplitude features (that is just x and z are used for Hurwitz). This must be added for a more apples-to-apples comparison between the two methods. Even if it is noisy, the authors should add peak-to-peak estimates to Hurwitz’s feature set.\n2)\tRe: clustering it would be useful to know how clustering using x, y, and alpha compare with clustering using typical waveform features (e.g. PCA decomposition of the time-course). Directly demonstrating that including position information aids unsupervised clustering would be useful here. Would, for example, someone using Kilosort want to add these position features for clustering neurons?\n\nMinor:\n\n1)\tDoes the neural network denoising used by the authors induce any distortion of the spike waveforms in these recordings?   \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, no issues here.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "PlWOJOnTys2", "reviewer": "Reviewer_PPLm", "summary": "The authors describe a pipeline for processing recordings from neuropixel probes which are the current state of the art in in-vivo recordings from single neurons in the brain. ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors describe a pipeline for processing recordings from neuropixel probes which are the current state of the art in in-vivo recordings from single neurons in the brain. Their pipeline consists of (a) a point neuron model for spike localization in 3D, (b) a denoising step for the waveforms and the resulting localizations, (c) motion registration and (d) spatial clustering. This is an interesting approach, but the way the paper is presented this seems more like a description and validation of a toolbox than a machine learning application paper. Also, step (b) is simply reused from another paper (which is totally fine, but hard to sell as a contribution then). \n\nAn additional issue with the paper is that in many instances (e.g. the denoising step for the waveforms and the images) the result is not really validated and only very briefly explained. Why is it “correct” to remove all the events in line 2 of the figure to the right? I suppose these are collisions, but isn’t it dangerous to remove them altogether? Weren’t real spike the cause of these as well?\n\nFor the image denoising, how does it lead to different results than simply smoothing the image with a Gaussian kernel? The authors state in line 98 a variety of methods they actually could use for denoising, which was it? Does the choice of method influence the result? Finally, I am not convinced taking the square root of count data and applying Gaussian denoising should be called Poisson denoising. \n\nThe remaining results look very promising, yet extremely dense for such a conference paper. Therefore, I would recommend a more extended format at a neuroinformatics/comp neuro journal venue. After considering the rebuttal and the discussion, I am still not convinced that this form of paper with extensive supplement is ideal for NeurIPS, but can see the point that it is within scope.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations were only superficially discussed, but I don't see negative societal impact of this work. The authors could add a discussion on potential improvements.", "review_text": "", "extra_scores": {"ethical_concerns": "none.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work brings together a number of algorithms to provide a method to detect the 3D location of cells from data collected with new high-density electrode arrays: Neuropixels. All reviewers appreciated the analysis developed by the authors in this work. The main discussion centered around the scope of NeurIPS with respect to engineered solutions to neural imaging problems. The authors, as well as some reviewers, pointed out the historical presence of such papers at NeurIPS, as well as the stated scope. Thus I am happy to recommend this paper for acceptance at NeurIPS.
Decision: Accept (Poster)