dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements Financial support from the  IHU Mediterranee-Infection , Marseille, France; CookieTrad for English corrections of the manuscript are gratefully acknowledged. This work was supported by the  French Government  under the  "Investments for the Future" program  managed by the  National Agency for Research (ANR)  and the  IHU Mediterranee-Infection  (reference  10-IAHU-03 ),  Marseille, France.
, 2001 , Luque et al. , 2019). Computationally, (i. e. , tonic firing mode, see  Luque et al. , 2019). PCs also integrated the error-related   (Clopath et al. , 2014 , Badura et al. , 2016) , whereas the remaining synaptic connections in the model were either non plastic or electrical, as between IO cells (Fig. 1C ; see Methods). We assessed cerebellar-dependent r-VOR adaptation by means of a 1 Hz sinusoidal and MF-MVN synaptic efficacies (which were randomly initialised) to adapt the r-VOR gain such as to reduce the error signalled by IO cells. After about 1000 s, the r-VOR gain (averaged over 40 simulated individuals) plateaued at 0. 95 (Fig. 2A ), a value consistent with experimental VOR data in humans under 1 Hz sinusoidal head rotations  (Dits et al. , 2013). Retina slip errors, as encoded by firing of IO cells, decreased as VOR accuracy improved (Fig.
The membrane capacitance of model PCs decreased as a function of age  (Li and Li, 2013)  (Fig. 4B ; see Methods). This led to an increase of tonic firing rates in older PCs (Fig. 4C ), consistently with electrophysiological data  (Zhang et al. , 2010). We also assessed the relation between the duration of post-complex spike pauses and the duration of precomplex spike ISIs in model PCs. We realised this measure by incrementally increasing the PF inputs whilst maintaining the CF stimulation constant (i. e. , only ISIs immediately as a function of age (Fig. 3E ), in contrast to actual epidemiological data (e. g. ,  Li et al. , 2015). following complex spikes were considered for this analysis, as in experimental data, that the linear relation between the duration of post-complex spike pauses and the duration of pre-complex spike ISIs would be preserved during ageing (Fig.
Finally, as shown in Eq. 33, the amount of LTP at PFs was fixed  (Kawato and Gomi, 1992 , Luque et al. , 2016 , Luque et al. , 2011a) , with an increase of synaptic efficacy equal MF-MVN synaptic plasticity. The LTD/LTP dynamics at MF-MVN synapses were the computation suitable for an event-driven simulation scheme as EDLUT  (Naveros et al. , same     2 x k x e cos x   (37) Note that there is no need for sensorimotor delay compensation thanks to the previous learning rule ( LTD  in Eq. 32). This second STDP mechanism accounted for learning (producing LTD in the activated MF-MVN synapses upon the arrival of the instructive signal and LTP otherwise). Well-timed sequences of increasing/decreasing levels of MVN activation ultimately shaped the cerebellar output during VOR adaptation. The EDLUT source code is available at the following URL: www.
This method is able to reach human performance and offers a better F1 score (harmonic mean of sensitivity and precision) than CaImAn to infer neural activity in the developing CA1 without any user intervention. It also enables automatically identifying activity originating from GABAergic neurons. Overall, DeepCINAC offers a simple, fast and flexible open-source toolbox for processing a wide variety of calcium imaging datasets while providing the tools to evaluate its performance. Significance statement: Inferring neuronal activity from calcium imaging data remains a challenge due to the difficulty in obtaining a ground truth using patch clamp recordings and the problem of finding optimal tuning parameters of inference algorithms. DeepCINAC offers a flexible, fast and easy-to-use toolbox to infer neuronal activity from any kind of calcium imaging dataset through visual inspection.
Introduction In vivo calcium imaging is widely used to study activity in neuronal microcircuits. Advances in imaging now allows for the simultaneous recording of several thousands of neurons  (Stringer et al. , 2019). One difficulty resides in how to infer single neuron activation dynamics from changes in fluorescence of a calcium indicator. A challenge is therefore to offer an analytical tool that would be scalable to the wide variety of calcium imaging datasets while providing reliable analysis. State of the art computational tools to infer neuronal activity (such as CaImAn  (Giovannucci et al. , 2019; Pnevmatikakis et al. , 2016) ) are based on the deconvolution and demixing of fluorescence traces from segmented cells. However, in order to optimize the deconvolution parameters, a ground truth based on simultaneous targeted patch-clamp recordings and two-photon imaging is necessary  (Chen et al.
, 1989; Galli and Maffei, 1988; O'Donovan, 1989; Provine, 1972) , which results in frequent cell coactivations, 2-the somata of pyramidal neurons are densely packed which results in spatial overlap, 3-Different calcium kinetics are observed in the same field of view (due to different cell types and different stages of neuronal maturation  (Allene et al. , 2012) ). All these points are illustrated in Videos 1 and 2 (Region CA1 of the hippocampus from mouse pups). In addition, most methods do not offer solutions to evaluate the performance of neuronal activity inference on user datasets. To meet those challenges, we have developed a graphical user interface (GUI) that allows for such evaluation through data exploration and a method based on deep-learning to infer neuronal activity. Even if several deep-learning-based methods to infer neuronal activity from fluorescence signals have already been developed  (Berens et al.
To our knowledge, no calcium imaging datasets from the developing hippocampus in vivo with simultaneous electrophysiological ground truth measurements are available. The most accurate ground truth would require targeted patch-clamp recordings with two-photon imaging on all the different hippocampal cell types with different calcium dynamics. This is technically difficult, time consuming and even more during development as the ground truth must be obtained from cells at various stages of maturation. As a result, we decided to base the ground truth on the visual inspection of raw movies using a custom-made GUI. It gives the advantages to work on any kind of calcium imaging dataset and to offer an easy tool to benchmark methods that infer neuronal activity. The GUI offers a tool to precisely and manually detect all calcium transients (from onset to peak, which is the time when cells are active).
We collected and combined a corpus of manual annotations from four human experts representing 37 hours of two-photon calcium imaging from 11 mouse pups aged between 5 to 16 postnatal days in the CA1 region using GCaMP6s. Almost 80 % of the labeled data was used to train the model, while the rest was kept to benchmark the performance. Then, movies were processed using a convolutional neural network with an attention mechanism and a bidirectional long-short term memory network  (Hochreiter and Schmidhuber, 1997; LeCun and Bengio, 1995; Vaswani et al. , 2017). To evaluate the method, we used the ground truth as a benchmark. We found that this method reached human level performance and offered a better sensitivity and F1 score than CaImAn to infer neuronal activity in the developing hippocampus without any user intervention.
Methods In this section, we will describe all the necessary steps to build a deep learning neural network "DeepCINAC". This toolbox was developed to analyze in vivo two-photon calcium imaging data acquired in the developing hippocampus (See § Experimental procedure and data acquisition). As a first step, we needed to set a ground truth that was established on the visualization of the recorded movie by three to four human experts ( § Ground truth). Then data are pre-processed ( § Data pre-processing and feature engineering and model description) and used to train the network ( § Computational performance). As a final step, we used labelled data to evaluate the performance of DeepCINAC ( § Performance evaluation). Tutorials and the source code are freely available online ( § Toolbox and data availability).
Data visualisation: GUI To visualize our data and explore the results from any spike inference method, we designed a graphical user interface (GUI) that provides a visual inspection of each cell's activity (Figure  2 ). The GUI offers a set of functionalities allowing visualization of i) calcium imaging movies centered and zoomed on the cell of interest during a time window that includes a given transient, ii) sources, transient profiles and their correlations (as developed by Gauthier and collaborators), iii) transient fluorescence signal shape. Additionally the GUI can be used to i) display the spike times from an inference method (Figure  2A1 -2), ii) establish a 'ground truth' (Figure  2B ), iii) visualize DeepCINAC predictions (Figure  2C ). The GUI was developed using Python and Tkinter package.
2B: The GUI can be used to establish a  'Ground Truth'. In   Activity inference : Electrophysiological ground truth. Ground truth data from experiments previously described were taken from crcns. org. (Chen et al. , 2013; GENIE Project, 2015). Briefly, visual cortex neurons expressing the calcium indicator GCaMP6s were imaged while mice were presented with visual stimuli. 60 Hz two photon imaging and loose cell-attached recordings at 10 kHz were performed simultaneously. Using ImageJ software, we downsampled imaging data to 10 Hz by averaging every six frames and rescaled it to 1. 2 µm/pixel. We considered a cell active during a rise time if a spike was detected during that time, and used the previously described GUI to convert those data in the cinac format so we could produce benchmarks and train a classifier using those data (see Table  1  and Table  S1  for more details).
After training our classifier on a first set of cells, we used the predictions obtained on new data to establish additional ground truth based on the mistakes made on those data. At least two human experts labeled segments of 200 frames containing the wrong predictions. Additional visual ground truth was established by one human-expert (RD) on three other datasets from our lab using the GUI: i) GCAMP6s calcium imaging movies from the developing barrel cortex ('Barrel-ctx-6s', 1. 5Hz, 1. 2µm/pixel)  (Modol et al. , 2019)  , ii) GCaMP6m imaging movies ('Hippo-6m', 10 Hz, 2 µm/pixel) and iii) GECO imaging movies ('Hippo-GECO', 5 Hz, 2 µm/pixel) both from the adult hippocampus (see Table  S1  for the details). For 'Barrel-ctx-6s' and 'Hippo-GECO', the CaImAn spike inference had already been performed by the original experimenter.
To overcome the issue of the imbalanced data and to enlarge the dataset, we used the following three approaches: #1: Data augmentation  (Perez and Wang, 2017) : temporal and spatial data augmentation were used. Temporal augmentation was used in that each block of 100 frames was overlapping with each other using a sliding window of 10 frames of length. Spatial augmentation took the form of transformations such as flip, rotation or translations of the images. The data augmentation was done online, meaning that the transformations were done on the mini-batches that the model was processing. This allowed avoiding memory consumption and generating a dataset on multiple cores in real time. #2: Simulated data: To balance our dataset, and increase the ability of the network to predict a fake transient as false, we have simulated calcium imaging movies with a higher rate of overlapping activity than our dataset (an example of artificial movie is available online on the gitlab page, alongside the source code: https://gitlab.
We finally added some gaussian noise on every frame. #3: Data stratification: In order to balance the data, we used data augmentation on selected movie segments (underrepresented segments), and excluded others (overrepresented segments) from the training data set. After data stratification, we obtained approximately 60% of the movie segments containing at least one real transient, 30% at least one fake transient without real ones and 10% without transients. We were then able to be more precise over the proportion of segments with multiple transients or cropped transients. We gave higher weights to segments containing fake transients in order for the network to adjust the accuracy accordingly. The data augmentation, simulated data and stratification were used only to produce the training dataset and not the validation dataset.
Model description. We designed a joint model combining a forward-pass long-shortterm memory (LSTM), a backward-pass LSTM and convolutional neural network (CNN) features. In order for the bi-directional LSTM to focus on relevant information, we reinforced it by an attention process at the stage of encoding similar to previous work  (Bin et al. , 2018; Rémy, 2019). The model was designed using Python and Keras library  (Chollet, 2015) , see Figure  4 ). The model used to predict the cell activity takes three inputs, each representing the same sequence of 100 frames (around 10 seconds of activity). Each frame had dimensions of 25x25 pixels, centered around the cell of interest, whose activity we want to classify. The first input has all its pixels set to zero except for the mask of the cell of interest (cell activity).
Classifier training We trained the classifier on a Linux-based HPC cluster where 4 CPUs (Intel(R) Xeon(R) CPU E5-2680 v3), 320 GB of RAM and 2 bi-GPU NVIDIA Tesla V100 were allocated for the processing task. To give an estimation of the time required to complete the training, the general classifier (CINAC_v1) was trained over 14 epochs. Training took around 40h (less than 3 hours by epoch).
Classifier prediction Using Linux-based workstation with one GPU (NVIDIA® GeForce GTX 1080), 12 CPUs (Intel Xeon CPU W-2135 at 3. 70 GHz), and 64 GB of RAM, the time to predict the cell activity on a movie of 12500 frames was on average 13 sec, approximately 3. 5 hours for a 1000 cells. The time to predict the cell type on a movie of 12500 frames was on average 2 sec, approximately 33 min for a 1000 cells. Similar performance was achieved using google colab.
-Performance evaluation Descriptive metrics for activity classifier: sensitivity, precision, F1 score. We evaluated the performance of the activity classifiers which predict for each frame if a cell is active or not. We chose to measure the sensitivity and precision values, as well as the F1 score that combines precision and sensitivity into a single metric defined as the harmonic mean of precision and sensitivity  (Géron, 2019). Because we have a skewed dataset (cells being mostly inactive), we chose not to use the accuracy. In order to compute the metrics, we needed to evaluate the number of true negative and false negative transients. To do this evaluation, we defined all putative transients. We identified the putative transient using the change of the sign of the first derivative on a smooth fluorescence time-course to detect all onsets and peaks.
We chose to measure the sensitivity and precision values, as well as the F1 score. To do so we used the metrics module of the Python package scikit-learn  (Pedregosa et al. , 2011 ) that returns the confusion matrix and a classification report containing those metrics. Statistical analysis. The distribution of F1 score values on the datasets for each inference method were compared using Wilcoxon signed-rank test with an a priori significance level of p = 0. 05 using scipy Python package  (Oliphant and Jones, 2001). This test was Comparison with CaImAn. We compared the classifier performance against a state of the art computational tool, namely CaImAn. To fairly compare CaImAn and DeepCINAC to the 'Ground Truth' we used the cell contours obtained from the CNMF. The spike inference from the MCMC as well as, DeepCINAC predictions and the 'Ground Truth' were established on these contours.
-Toolbox and data availability The source code is available on gitlab (https://gitlab. com/cossartlab/deepcinac). The page includes a full description of the method, a user manual, tutorials and test data, as well as the settings used. A notebook configured to work on google colab is also provided, allowing for the classifier to run online, thus avoiding installing the necessary environment and providing a free GPU. The toolbox has been tested on windows (v7 Pro), Mac Os X (MacOS Mojave) and Linux Ubuntu (v. 18. 04. 1).
6C. Boxplots showing F1 Score for the 3 human experts (RD, JD, MP) and CINAC_v6 evaluated against the known ground truth from 4 cells from the GENIE project. Each colored dot represents a cell. Cell labels in the legend correspond to session identifiers from the dataset. CINAC_v6 is a classifier trained on data from the GENIE project and the 'Hippo-dvt' dataset (Table  1 , Table  S1 ).
DeepCINAC performance evaluation on developing hippocampus dataset Comparing DeepCINAC against CaImAn and human level. We compared the performance of DeepCINAC and CaImAn  (Pnevmatikakis et al. , 2016) , a well established algorithm to infer neuronal activity, against the visual ground truth on CA1 hippocampus data during development ('Hippo-dvt'). We first evaluated DeepCINAC ( CINAC_v1) on 20 putative pyramidal neurons and 5 interneurons (Figure  7 ). The median sensitivity was 80. 3% (interquartile range 75-94. 5, Figure  7A ), the median precision was 90. 8% (interquartile range 81. 2-95. 5, Figure  7B ) and the median F1 score was 86. 3% (interquartile range 78. 9-91. 3, Figure  7C ). We next evaluated CaImAn on the same cells using the same metrics. The median sensitivity was 60. 6% (interquartile range 45.
6-76, Figure  7A ), the median precision was 100% (interquartile range 93. 8-100, Figure  7B ). The median F1 score was 70. 1% (interquartile range 62. 6-81. 6, Figure  7C ), which was significantly lower than CINAC_v1 F1 score (Wilcoxson signed rank test, T=50 and p=0. 002). Finally, we asked if DeepCINAC could perform as well  Each colored dot represents a cell, the number inside indicates the cell's id and each color represents a session as identified in the legend. CINAC_v1 is a classifier trained on data from the 'Hippo-dvt' dataset (Table  1 , Table  S1 ). CINAC_v1 is a classifier trained on data from the 'Hippo-dvt' dataset (Table  1 , Table  S1 ).
DeepCINAC performances on other datasets A major aspect to consider in the development of algorithms to infer neuronal activity from calcium imaging data, is the ability to be easily scalable to the wide variety of datasets (i. e. different indicators, different brain regions,. ). We investigated the extent to which DeepCINAC (CINAC_v1), that was trained on data from the developing hippocampus, would perform on other datasets (Figure  8-1 ). To answer that question, we used i) GECO imaging movies ('Hippo-GECO', 5 Hz, 2 µm/pixel, Figure  8 -1A), ii) GCaMP6m imaging movies ('Hippo-6m', 10 Hz, 2 µm/pixel, Figure  8 -1B) both from the adult hippocampus, iii) GCAMP6s calcium imaging movies from the developing barrel cortex ('Barrel-ctx-6s', 1. 5Hz, 1. 2µm/pixel, Figure8-1C)  (Modol et al. , 2019)   To overcome these poor performances on 'Hippo-GECO' and 'Visual-ctx-6s' and to improve performances on 'Barrel-ctx-6s' and ''Hippo-6m' datasets, we considered two strategies.
The first one consists in training a classifier specific to the data. The second one consists in adding part of the new data to the large database to improve the classifier ability to generalize (Figure  8  and 8-1 ). Since our performances were low using CINAC_v1, we adopted the first strategy to improve the classifier on 'Hippo-GECO' and 'Visual-ctx-6s' datasets. We used part of these datasets to train specific classifiers and evaluate them on the remaining data (Table  S1 ). First, we observed that the 'Hippo-GECO' specific classifier (i. e. CINAC_v3) performed better than On the datasets from the adult hippocampus ('Hippo-6m') and developing barrel cortex ('Barrel-ctx-6s'), since CINAC_v1 performances were close to the performance reached on 'Hippo-dvt' dataset, we considered the second strategy.
We extended the CINAC_v1 training dataset with labeled data from 'Hippo-6m' and 'Barrel-ctx-6s' datasets and trained a new classifier (CINAC_v4). First, we observed that on the 'Hippo-6m' dataset, CINAC_v4 classifier has better sensitivity and precision than CINAC_v1 and CaImAN (CINAC_v1 median sensitivity=53. 8%, CINAC_v4 median sensitivity=58. 7%, CaImAn median sensitivity=37. 3%, CINAC_v1 median precision=95. 3%, CINAC_v4 median precision=96. 2%, CaImAn median precision=95. 7%, CINAC_v1 median F1 score=68%, CINAC_v4 median F1 score=71. 6%, CaImAn median F1 score=51. 4%, Figure  8B ). Second, we found that on 'Barrel-ctx-6s' dataset, CINAC_v4 classifier performed better than CINAC_v1 and CaImAn (CINAC_v4 F1 score=87. 3%, CINAC_v1 F1 score=79. 7%, CaImAn F1 score=53. 3%, Figure  8C  Because we benefited from already published data from our group ('Barrel-ctx-6s'), we next asked if we could arrive at the same conclusion using CINAC_v1.
We used the activity inferred by CINAC_v1 on this data and performed assemblies detection analysis as described previously  (Modol et al. , 2019). We found the same number of assemblies as the original analysis (Figure  8 -3A and 8-3B, first two panels), as well as the same topographic organization (Figure  8 -3A and 8-3B, bottom panels). We confirmed that the assemblies detected by either CaImAn or CINAC_v1 were composed of similar cells. GECO' dataset (Table  1 , Table  S1 ). 8B: Boxplot displaying the sensitivity (top panel), precision (middle panel) and F1 score (bottom panel) for 'Hippo-6m' dataset. For each panel, we evaluated CaImAn performance as well as two different versions of  CINAC (v1 and v4). CINAC_v1 is a classifier trained on data from the 'Hippo-dvt' dataset and CINAC_v4 is a classifier trained on data from the 'Hippo-dvt  ', , Table  S1 ).
8C: Boxplot displaying the sensitivity (top panel), precision (middle panel) and F1 score (bottom panel) for '  For each panel, we evaluated CaImAn performance as well as two different versions of  CINAC (v1 and v4). CINAC_v1 is a classifier trained on data from the 'Hippo-dvt' dataset and CINAC_v4 is a classifier trained on data from the 'Hippodvt  ',   S1 ). Each colored dot represents a cell, the number inside indicates the cell's id and each color represents a session as identified in the legend. Each colored dot represents a cell, the number inside indicates the cell's id and each color represents a session as identified in the legend. CINAC_v1 is a classifier trained on data from the 'Hippo-dvt' dataset (Table  1 , Table  S1 ). CINAV_v1 is a classifier trained on data from the 'Hippo-dvt' dataset, CINAC_v5 is a classifier trained on data from 'Visual-ctx-6s' dataset, CINAC_v6 is a classifier trained on data from 'Visual-ctx-6s' dataset and 4 cells from the 'Hippo-dvt' dataset (Table  1 , Table  S1 ).
Cell type inference using DeepCINAC Recently a deep-learning method using a similar model to DeepCINAC was proposed to differentiate cell types  (Troullinou et al. , 2019). This model was based on the analysis of fluorescence traces from various cell types and automatically classified imaged cells in different types. We asked whether DeepCINAC would be able to distinguish interneurons from pyramidal cells using as input the calcium imaging movie rather than the fluorescence trace. Additionally we added a noise category in the training dataset allowing us to automatically discard cells. We achieved a general F1 score of 86%. We had a sensitivity of 90. 2%, 81. 6% and 81. 8% and a precision of 90. 2%, 91. 2% and 60% for pyramidal, interneuron and noisy cells respectively (Table  2 ). Since activity inference performance using DeepCINAC depends on the cell type, we perform this cell type prediction before activity inference.