dataesr/scientific-paragraphs-categorization huggingface dataset:
This improves the average performance of the classification. However, it introduces some time delay to accumulate enough evidence to make a prediction. We also report the accuracy values for the classification post hoc with a k-nearest neighbors algorithm on the activation histograms. Two results are shown in figure  5 , one for the original HOTS and another with the homeostatic gain control for the clustering phase: HOTS with homeostasis. As expected, for all datasets and both modalities, the accuracy of the online classification improves as the number of events increases. Within a dataset, the total number of events for the samples can vary. We set a maximum number of events to represent the accuracy by taking the 90th percentile of the dataset in terms of number of events. The accuracy of the eventbased classification for each dataset is shown in Figure  5.
Note that the x-axis is plotted on a logarithmic scale and that only a small number of events will allow for significant classification above chance. In Figure  5 -(a), we observe the online inference for the Poker DVS dataset. The RESULTS PokerDVS. ipynb notebook reproduces the results and figures for this dataset. The post hoc methods perform well but do not reach 100% accuracy, with an advantage for clustering with homeostasis (95. 0% accuracy) than without (85. 0% accuracy). For online HOTS, the accuracy quickly reaches 100% after only an average of 19. 4% of the total number of events, i. e. one fifth of the total event stream. This online classification allows an ultra-fast categorization of objects in terms of events: only a few events are needed for the classification to reach a good level of accuracy.
If we set a confidence threshold on the MLR, we obtain a perfect classification once at least 35 events are received. Given the small number of samples in this dataset, we evaluate the performance of the network on two more complex and widely used datasets. The online accuracy on the N-MNIST dataset is shown in Figure  5-(b ) and reproducible at RESULTS NMNIST. ipynb. The original algorithm already performs well with the classification by histogram comparison, reaching 94. 4% for HOTS and 92. 4% for HOTS with homeostasis. In this particular example, the homeostatic gain control did not improve the performance for the clustering phase, and we recall that the main advantage of this regularization is to reduce the sensitivity of the unsupervised learning to the initialization  (Grimaldi et al.
, 2021). For online HOTS, we observe an accuracy above chance after the very first events, which increases with the number of events received by the network. Note that the accuracy value increases drastically after about 1000 events and reaches values  above the original method approximately at 2000 events, the average of events for the N-MNIST dataset being 4176 events (see DATASET STATS. ipynb). If we compute the mean performance over all the decisions, i. e. for each event, we get an accuracy of 70. 1% and 96. 6% for the accuracy calculated when the decision is made at the timestamp of the last event. Another way of calculating the post hoc accuracy with this probabilistic approach is to choose the decision that was made with the highest confidence. This gives us an accuracy of 97. 4%, which is close to the SOTA (see next section).
, 2016; Frenkel et al. , 2020; Thiele et al. , 2018). For the DVSGesture dataset, we confirm the improvement of our method over the original one on more realistic event-based recordings (see Figure  5-(c )). For these more complex gesture recognition tasks, the online HOTS accuracy remains close to the chance level for about 100 events. More evidence needs to be accumulated, and then the accuracy increases monotonically, outperforming the previous method after about 10. 000 events (an average of 9. 3% of the sample). These event-based recordings have a much higher event density than the other datasets, and we remind the reader that only 3 seconds of the recording is kept to test our algorithm. The average accuracy for all the decisions is 85. 7% and 87. 2% when the decision is taken at the last event received.
Comparison to the state-of-the-art To compare the performance of our method with the SOTA, we choose to compute the accuracy when the decision is made with the highest confidence, as other methods do not present the event-driven online accuracy in their results. We report a table of the best accuracy results found in the literature for the N-MNIST and the DVSGesture datasets. All methods mentioned in 2. 2 are not reported here, preprints are discarded, and we focus on event-based methods that achieve the best performance. We split the table 2 into two different parts for the methods that N-MNIST DVS Gesture HOTS (with k-NN)  (Lagorce et al. , 2017)  94. 39% 83. 0% HATS  (Sironi et al. , 2018)  99. 1% -SLAYER  (Shrestha and Orchard, 2018)  99. 2% 93. 64% Spike-based BP  (Fang et al. , 2021)  99.
61% 97. 57% DSNN-STDP  (Thiele et al. , 2018)  95. 77% -DECOLLE  (Kaiser et al. , 2020)  96% 95. 54% self-BP  (Zhang et al. , 2021)  -84. 76% hybrid CNN-SRNN  (Yin et al. , 2021)  -97. 61% Ours 97. 4% 89. 8%  are biologically plausible (bottom) and the others (top). We skip the comparison of the results obtained with the Poker DVS dataset, which serves as a toy model but does not provide a challenging classification task. We argue that, even if we don't outperform these SOTA results, this simpler 3-layer feedforward network structure with a bio-plausible learning achieves very competitive accuracy values. In addition, our classifier is the first to provide always-on decision making.
Robustness to jitter We also wanted to assess the robustness of this event-driven object recognition method. To do this, we perturb the original datasets by adding temporal or spatial jitter to the events. Jitter is applied only to the test set to add noise to the signal used for classification. As described in section 2. 1, we use the tonic package to apply temporal or spatial jitter to the test samples. For each amount of jitter applied to the test set, 10 repetitions are performed to obtain different accuracy values. Finally, we fit a beta distribution to each of these results to compute the percentiles shown in Figure  6. To compare with previous results obtained in  Grimaldi et al. (2021) , we plot the offline accuracy obtained when making one decision per sample. We reduce the number of computations for this analysis by using subsets of the N-MNIST dataset (1000 samples).
For each amount of jitter applied to the test subset, 10 repetitions are performed to obtain different accuracy values. Finally, we fit a beta distribution to these results to compute the percentiles shown in Figure  6. As the proposed method is a proof of concept for event-based computation, the simulations on GPU are not optimized and results with jitter applied to the DVSGesture dataset are not computed for reasons of simulation time. We highlight the fact, that to our knowledge, no other studies have performed this test on event-based recordings. Simulated on two different DVS datasets, these results provide insight into the robustness of our algorithm, but also highlight the features that are relevant for classification within the event-based recordings. As expected, the higher the jitter, the greater the negative impact on classification.
The decrease in accuracy as a function of jitter fits well to a sigmoid function that decreases from a maximum accuracy value to reach the chance level. Using this fit, it is possible to define a critical standard deviation of jitter in pixels or in ms where the accuracy drops to half of its maximum compared to the chance level. This half saturation level provides a signature value for the relevant information contained in the signal. Figure  6-(a)  shows the evolution of the accuracy for the different methods as a function of the amount of spatial jitter applied to the PokerDVS dataset. Accuracy reaches half sat-uration for a spatial jitter with a standard deviation of 1. 50, 1. 66 and 3. 87 pixels, for HOTS, HOTS with homeostasis and online HOTS respectively. Figure  6-(b)  shows results for the N-MNIST dataset.
Curves for HOTS, HOTS with homeostasis are very close but show slightly different half saturation levels: 2. 42 pixels for HOTS, 2. 32 and for HOTS with homeostasis. The homeostasis itself does not give significant improvement in this particular example, in line with the results obtained in Figure  5-(b ). However, online HOTS shows overall significantly better performances and reaches its half saturation level at 3. 74 pixels Reaching half saturation level at approximately a standard deviation of the spatial jitter equal to 3. 8 pixels demonstrates that this method relies heavily on the spatial information. However, considering the small pixel grid of these datasets (32 × 32 for PokerDVS and 28 × 28 for N-MNIST) and the gradual decrease of the accuracy, we observe robustness to spatial jitter for the different methods.
In any case, the results demonstrate a significant improvement of the robustness to spatial jitter for the new method. Panels (c, d) in Figure  6  illustrate a high resilience of the network to temporal jitter, note that the x-axis is composed of log 10 -spaced values. The average recording time for the Pok-erDVS dataset is 7. 1 ms and 308 ms for N-MNIST. For Pok-erDVS (see figure  6-(d )), we obtain the following half saturation levels corresponding to one standard deviation of the jitter distribution in ms. For HOTS: 1. 77 ms; HOTS with homeostasis: 8. 16 ms; and online HOTS: 34. 5 ms. For N-MNIST (see figure  6-(d )), the half saturation values are 49. 77 ms, 41. 32 ms and 126. 2 ms for HOTS, for HOTS with homeostasis and for the algorithm presented in this study respectively. Even the original method offers a high resilience to temporal jitter compared to the duration of the recordings.
Acknowledgment Antoine Grimaldi  and  Laurent Perrinet  received funding from the  European Union   ERA-NET CHIST-ERA 2018 research and innovation program  under grant agreement  N°ANR-19-CHR3-0008-03  (" APROVIS3D "). Sio-Hoi Ieng ,  Ryad Benosman  and  Laurent Perrinet  received funding from the  ANR  project  N°ANR-20-CE23-0021  (" AgileNeuroBot "). During the preparation of this work the authors used DeepL Write in order to improve language and readability. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.
Analysis of the Fluorescence Lifetime microscopy images NAD(P)H intensity images were obtained using Fiji-ImageJ (NIH, Bethesda, MD, USA) as the sum of the FLIM stack. All FLIM data were processed and analyzed with a custom written MATLAB (Mathworks, Natick, MA, USA) software. FLIM data were transformed from time domain to frequency domain by using a fast Fournier transform and plotted in the phasor plot as previously described  (Stringari et al. , 2011; Ranjit et al. , 2018). As our FLIM data were acquired in the time domain (Fig. 1C ) we transformed the multi-exponential fluorescence intensity decay in every pixel of the image into a point in the 2D phasor plot of coordinates g and s (Fig. 1E ) with the following equations: = ∫ ∫ (1) = ∫ ∫ (2) where i and j indicate the order pixel of the image and ω is the frequency.
We implemented single cell analysis to quantify the cellular mean of the fraction of bound NAD(P)H from the 3D FLIM data by using masks of single cells defined by the LifeAct-mCherry intensity for the stable cell line and by the NAD(P)H intensity for the primary T cells (Fig. S2A ). First, we performed a maximum intensity Z-projection across all the Z planes of the 3D stack (Fig. S2A ). Then, binary masks were created using a gaussian blur and a threshold to identify and segment each single cell (Fig. S2B ) using Fiji-ImageJ (NIH, Bethesda, MD, USA). The same mask was used for every Z plane of one ROI rather than constructing separate masks for each Z plane. With a MATLAB custom written code, we also applied a threshold (5 photons) on the NAD(P)H intensity to consider only the cell signal and exclude the background at each Z plane.
4C , Activated) shows asymmetry between the cell side closer to the slide (the surrogate immune synapse) and its opposite, with a fraction of bound NAD(P)H that is higher at the immune synapse than in the top of the cells. This asymmetry suggests that there is relatively more OXPHOS at the synaptic side than in the rest of the cell, whereas in resting cells the distribution of fraction of bound NAD(P)H is symmetric (Fig. 4C , control). This might be due to the polarized distribution of mitochondria at the immune synapse that was reported  (Quintana et al. , 2007; Baixauli et al. , 2011; He et al. , 2019)  and that we observed in our model (Fig. S9 ). Altogether, these data show that 2P-FLIM of NAD(P)H in T cells can reveal temporal and spatial dynamic changes in their metabolism at a single cell level and in subcellular compartments.
Spreading of primary T cells was more stable overtime than spreading of Jurkat T cells, and the area of primary T cells plated on activating antibodies was always significantly bigger than the area on cells plated on Poly-L-Lysine alone (Fig. 6A ). Moreover, there was a more convincing negative correlation between fraction of bound NAD(P)H and area of the primary T cell at all time points tested (Fig. 6B  and data from individual donor in Fig. S10 ). Altogether, these results show that FLIM of NAD(P)H in T cells can reveal dynamic changes in their metabolism at a single cell level and can be combined with the analysis of other parameters, such as actin remodeling, that can be followed by fluorescence imaging.
Acknowledgements: We would like to thank  Xavier Lahaye  for providing us with peripheral blood mononuclear cells and the members of the "Integrative analysis of T  cell activation" (Institut Curie)  for support and discussions. We thank all the members of the "advanced microscopies" group at the aboratory for Optics and iosciences for helpful discussions and advice. We thank  Pierre Mahou  and  Emmanuel Beaurepaire  for scientific discussions and technical help.
Behavioral video data analysis Images of the animals' faces were acquired by video tracking at two possible rates: 60 Hz (mice 1-3) or 300Hz (mice 3-7). Images were processed with the DeepLabCut toolbox  (Mathis et al. 2018)  at the frame rate of acquisition. For the video tracking from below the snout, we first tagged 400 frames picked randomly across all datasets with the following tag: nose left, nose right, nose center, philtrum, teeth center, chin, tongue, front right whisker base, front right whisker tip , back right whisker base, back right whisker tip, left paw, and right paw. The neural network was trained to detect the markers on a dataset of 500 frames randomly selected among all videos. Through visual inspection, another 200 frames with unsatisfying tagging were manually labeled and added to the training dataset; the network was then re-trained with all labeled frames.
All videos were processed with the latter trained network. The marker positions measured in pixels and the likelihood of detection provided by DLC software were used to compute the movement of the different body parts. In each session, we computed standard deviation of x-y position of the nose and whisker markers (in pixels). These measures were ranked and averaged for each trial. We then identified the 5% trials with most and 5% trials with least face movements, 5 high (so trials with high movements), and 5 low (trials with the least movement) respectively. The idea behind this strategy stems from the observation that in every session there are trials with high levels of face motion and trials in which the animal remains still. These trials are in our view the most reliable way to normalize datasets across sessions and imaging parameters.
More than 95% neuropil correction factors computed this way were between 0. 5 and 0. 85, values below or above this range were respectively set to these boundaries. Finally single ROI fluorescence were Z-scored and a constrained deconvolution was applied  (Pachitariu et al. 2016)  to return a continuous spiking estimate. This preprocessing strategy and criteria were optimized from the freely available jRGECO1a dataset from the CRCNS website  (Dana et al. 2016 , Mohar et al. 2016)  to match the state of the art algorithmic performance in event detection  (Berens et al. 2018).
D, Behavioral performance in the stimulus space. Each point represents a set of two frequencies applied to the two whiskers. The diagonal represents the task boundary (F1 = F2). Proportion of left/right choices are color coded. E, Psychometric function drawn from conditions within the rectangle in D. (i. e. with a constant summed frequency F1 + F2 = 90 Hz) circles represent average performance and thin traces are logistic fits for individual animals (see methods). Black thick line represents a logistic fit for the pooled dataset. F, Three possible behavioral outcomes in one example session. Top: 3 possible behavioral outcomes: first lick right, first lick left, and no lick during the response window. Bottom: example session with behavioral outcome color coded. The trial index is a normalization of trial number between 0 and 1 (respectively first to last trial of the session).
Calcium imaging and further analysis were carried on a subset of sessions where the animal is considered expert (>70% Correct trials). D, Behavioral performance in the stimulus space. Each point represents a set of two frequencies applied to the two whiskers. The diagonal represents the task boundary (F1 = F2). Proportion of left/right choices are color coded. E, Psychometric function drawn from conditions within the rectangle in D. (i. e. with a constant summed frequency F1 + F2 = 90 Hz) circles represent average performance and thin traces are logistic fits for individual animals (see methods). Black thick line represents a logistic fit for the pooled dataset. F, Three possible behavioral outcomes in one example session. Top: 3 possible behavioral outcomes: first lick right, first lick left, and no lick during the response window.
Extreme trials were identified as trials with χj values more than 3σ from μ and were removed. After applying the inclusion and exclusion criteria described above, we analyzed the single unit responses. First, we subtracted the common average across channels from the raw ephys data to reduce noise. The data was then further preprocessed and spike sorted using Kilosort (https://github. com/MouseLand/Kilosort;  Pachitariu et al. , 2016) , with a spike detection threshold of -2 SD. The spike sorting results were manually curated using Phy (https://github. com/cortex-lab/phy). Spike clusters with high Kilosort quality scores that were stable across the session were included for analysis of single unit responses. We convolved the detected spikes of each unit with a Gaussian with an SD of 10 ms to derive a continuous estimate of the spike rate.
To estimate the receptive field of each neuron, we averaged the spikes that were evoked by each RF map checker in a time window in between 40 to 300 ms after checker onset. Given the variety of PSTH shapes, each neuron was assigned its own time window where the neuron showed increased or decreased spiking. We then fit a two-dimensional (2D)-Gaussian to estimate the width and center of both the ON and OFF RF. The quality of the fit was assessed using r 2 and a bootstrapped variability index (BVI), which estimated the reliability of the RF center estimate  (Kirchberger et al. , 2021). We resampled an equal number of trials as in the original dataset (with replacement) and regenerated the Gaussian fit. The BVI is defined as the ratio of the SD of the RF center position and the SD of the fitted Gaussian.
Acknowledgements We thank  Christiaan Levelt  for sharing experimental facilities and  Emma Ruimschotel  for genotyping. We further thank  Enny van Beest ,  Mehran Ahmadlou ,  Chris van der Togt  and  Ulf Schnabel  for sharing their expertise. J. L. C and J. A. H. were funded by  FLAG-ERA  grant  CHAMPmouse  through  de Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO).
The relative threshold r IES was selected by continuously exploring detected IES duration with respect to r IES (Supplementary section 2. 1 and Fig. S6 ). This approach is robust on the present dataset, and it would be interesting to evaluate the goodness of fit of our optimal relative threshold value r IES = 0. 7 on a bigger dataset. The heuristic could be adapted for real-time applications by evaluating the signal RMS(t) before the beginning of GA as opposed to during GA.
Logistic regression analysis A logistic regression approach with l 2 regularization  [54]  and a regularization constant C=1 were used to evaluate the contribution of several parameters to the prediction of significant time spent in IES. The criterion for class separation was chosen with the γ rebound phenomena, which happens for recordings with more than 30 seconds in IES (section 3. 3). The dataset was thus divided into a positive class (total time spent in IES is more than 30 seconds) and a negative class (total time spent in IES is less than 30 seconds). The dataset was separated into training and validation sets using a stratified group k-fold strategy: this strategy ensures that recordings from one individual are not split between the train and validation set (group) and that the validation sets of all folds have equivalent proportions of positive and negative labels (stratified).
Acknowledgments D. H. research is supported by a grant  ANR NEUC-0001 ,  CNRS  pre-maturation, and the  European Research Council (ERC)  under the  European Union 's  Horizon 2020 research and innovation program  (grant agreement No  882673 ). Experimental work in the J. T. P. lab was supported by  Gladstone Institutes. We thank  Kathryn Clairborn ,  Christophe Sun , and  Matteo Dora  for critical feedback on our manuscript.
It is important to note that in cases involving very large data sets (e. g. , Alzheimer's Disease Neuroimaging Initiative (ADNI) data sets), it may not be feasible to QC all of the collected scans for segmentation errors. As such, at least a subset of the data set should be QC-ed in order to get a sense of the data quality. Similar to the judgments of MR image quality, the manual evaluation of segmentation errors is also subjective; thus, it requires standardized approaches to promote consistent decisions. There are several resources available to provide some options for standardized protocols: 1) MAGeT-related QC (https://github. com/CoBrALab/documentation/wiki/MAGeT-Brain-Quality-Control-(QC)-Guide), 2) HippUnfold's automated QC (DSC overlap with a deformable registration), and 3) MRIQC  (Esteban et al.
While this approach reduces the burden of QC of segmentations, it does require QC of the MRI scans at each timepoint separately and of the pairwise registrations. However, QC of pairwise registrations can be done using a semi-automatic method by examining registration quality metrics (e. g. , intensity crosscorrelation in the MTL region after registration) and focusing manual QC procedures on image pairs where that metric is more than 2. 5 standard deviations from the mean  (Xie et al. , 2020). It is important to note that although there are differences in the approach for longitudinal segmentation, the QC procedures we have reviewed above apply similarly to data at all timepoints  (Shaw et al. , 2020). When datasets are large, the general quality of segmentation and potential extent of segmentation errors can be approximated from the procedures applied to a subset of cases that are selected at random.