dataesr/scientific-paragraphs-categorization huggingface dataset:
Besides, The Spike Layer Error Reassignment (SLAYER) training algorithm  (Shrestha and Orchard, 2018)  is successfully deployed to the SCRNN for the purpose of generalization and training stability. It utilizes both temporal error and axonal delay credit assignment to minimize the computational complexity. The use of SLAYER effectively prevents the common gradient vanishing and explosion problem associated with recurrent neural networks. Since the recurrent propagation between the SCRNN cells relies on the information fusion from inputs of current timestamps and output from previous timestamps. Particularly for SCRNN, a spiking feature map integration method is developed in the SCRNN cell to maintain information continuity in the temporal domain. Furthermore, The SCRNN is validated by a series of experiments on the DVS gesture dataset  (Amir et al.
, 2017)  to prove its robustness for the motion-based neuromorphic action recognition problem. The remainder of this paper is organized as follows. Section 2 introduces the related work in the spiking recurrent neural network and SLAYER training algorithm. In section 3, detailed descriptions are provided in terms of individual SCRNN cell and overall SCRNN topology. The experiment results on the DVS gesture dataset is presented and discussed in section 4. The experiment result is analyzed and compared with previous work. Finally, the conclusions are provided in section 5.
, 2010; Tavanaei and Maida, 2017) , pattern recognition  (Han and Taha, 2010; Dhoble et al. , 2012; Mohemmed et al. , 2012; Kasabov et al. , 2013). Furthermore, many developed neuromorphic computing platforms have demonstrated tremendous potential in real-world power limited applications. The IBM TrueNorth systems consist of 5. 4 billion transistors with only 70mW power density consumption, which accounts for only 1/10,000 of traditional computing units  (Akopyan et al. , 2015). The SpiNNaker platform  (Furber et al. , 2013 (Furber et al. , , 2014) )  developed by Researchers at Manchester provides ASIC solutions to hardware implementations of SNNs. It utilized multiple ARM cores and FPGAs to configure the hardware and PyNN  (Davison et al. , 2009)  software API to enable the scalability of the platform.
Convolutional Recurrent Neural Network The convolutional recurrent neural network (CRNN) structure has been well studied in the second generation of ANNs. The convolution operation in the ANNs usually acts as a spatial visual feature extractor that assumes features are in different levels of hierarchy. The recurrent structure introduces memory to the network and an ability to deal with sequential data dependently. A significant design of the CRNN structure is the ConvLSTM structure  (Shi et al. , 2015)  that was initially designed for forecasting precipitation. By replacing the general gate activation by the convolutional operation, the network is able to exploit an extracted 3D tensor as the cell state. The ConvLSTM was also evaluated on the moving MNIST  (Srivastava et al. , 2015)  dataset and was shown to successfully separate the overlapping digits and predicted the overall motion with a high level of accuracy.
, 2017). Adopting a recurrent structure enables the neural network to encapsulate the global information while local features are extracted by the convolution layers. Yang et al. (2018)  demonstrated a Convolutional LSTM network that was successfully evaluated on various action recognition datasets. The importance of using CRNN structure in the application of human action recognition is that unlike action recognition in images, the same task in videos relies on motion dynamics in addition to visual appearance. Although CNNs and its variants like 3D convolution  (Ji et al. , 2013; Karpathy et al. , 2014)  achieves good performance, they still do not make sufficient use of temporal relations between frames. More recently, Majd and Safabakhsh (2019) designed a motion-ware ConvLSTM for the action recognition task which is an LSTM unit that considers the correlation of consecutive video frames in addition to the Spatio-temporal information.
In the SLAYER, this is done by convolving the spike trains in the kernel with a spike response kernel and followed by the threshold function. Each spike train will be transferred to the spike response signal then further to the membrane potential of the postsynaptic neuron. The process can be expressed as: a u,v (t) = s u,v (t) * σ (t) (2) u j,k (t) = K m=1 K n=1 W m,n a j+m-1,k+n-1 (t) + (s j,k (t) * ν(t)) (3) s j,k (t) = 1 & u j,k (t) = 0 when u j,k (t) ≥ V thr (4) where W denotes to the synaptic weights. u and v are the vertical and horizontal coordinate index of the input tensor. j and k represents the vertical and horizontal coordinate in the feature map. K represents the convolution kernel width and height. The 3D spiking convolution can decompose the input event based data into several spatio-temporal pattern feature maps, where each spike in the map corresponds to a specific pattern.
EXPERIMENT RESULTS In this section, the experimental result of action recognition using SCRNN will be presented. To validate the robustness of the SCRNN, we evaluated the network structure by performing the recognition task on the IBM DVS gesture dataset  (Amir et al. , 2017). The DVS gesture dataset comprises recordings of 29 different actors carrying out 10 different hand gesture actions. All recordings are captured by an Inilabs 128 × 128 dynamic vision sensor under three different lighting conditions. Each gesture sample has a duration of approximately 6 s. Figure  5  shows an example of hand waving gesture with 0. 5 s integral time interval in nature light condition. The goal is to classify the gesture event video data into a corresponding label. The DVS gesture dataset is split as 1,176 samples for training and 288 samples for testing as annotated.
It can be clearly seen from Figure  6B  that the actual spikes (shown in black) now have similar spike counts as target spikes (shown in red) for the input samples. It should be noted that, the target spikes and actual spikes have different spike timings but similar spike counts in each window. The experiment results are shown in Table  2 , where each listed architecture is simulated for 100 epoch over the full dataset. For each structure listed in the table, the accuracy is obtained by averaging the best testing accuracy among 5 repeated experiments with different random initialized weights. Among these experiments, the best testing accuracy of 10 class gesture is 96. 59% with the 3 layer SCRNN structure with the first convolutional layer consisted of 32 5 × 5 convolutional filters, second and third convolution layer has 64 and 128 3 × 3 convolutional kernels, respectively.
The l s is 50 ms which represents there are total 1,000/50 = 20 time steps. The loss and training curve for the best network structure is shown in Figures  7A, B. This structure also was used to train the 11 class gesture (plus a random other gesture action) and obtained a testing accuracy of 90. 28%. Thus, the loss can be very large at the start compared with normal loss value since the network can have an empty output with untrained weights and delays. It was found that setting the l s = 50ms produces the best result for SCRNN structure which can be explained as follows. First, the time resolution is matched with the frame continuity for this dataset, which means the individual segmented frame can either contain limited or redundant information with l s = 25ms or l s = 75ms. This can possibly weaken the connection between the frames from the perspective of recurrent convolutional operation.
Secondly, The confusion matrix in Figure  7C  shows a detailed performance of the SCRNN for the 10 gesture recognition tasks. Note that the amount of samples of arm roll is twice than other gestures in the original dataset. It can be seen that the SCRNN achieved an overall good performance except that the confusion between the hand-clapping and air drums gesture where there are totally 3 + 4 = 7 instances that SCRNN misclassified the hand clapping or air drum as each other. This is due to the dynamic similarity of these two gestures for some instances. Figure  8  demonstrates an example of misclassification which shows both 3D and 2D view of dynamics of these two gesture. From our observations, some of hand-clapping and air drum gestures exhibit a strong similar spike change pattern which is a potential reason that leads to misclassification.
This further matches our initial design purpose of SCRNN, which is an action dynamics sensitive, event stream pattern based recognition network. For comparison purpose, results from previously published work  (Amir et al. , 2017; Shrestha and Orchard, 2018; Wang Q. et al. , 2019)  on the IBM DVS gesture dataset is carried out which is shown in Table  3. It can be seen that the SCRNN approaches the state of the art recognition accuracy and surpassing the benchmark accuracy of IBM's work in 10 categories gesture classification tasks. The original work from IBM that running on TrueNorth was trained with Eedn  (Amir et al. , 2017)  and required extra filters and preprocessing before the CNN. On the other hand, the SCRNN takes the neuromorphic data directly from the sensor and the training process does not require any additional processing to the data.
The SLAYER algorithms  (Shrestha and Orchard, 2018)  using CNN with a feedforward structure achieved an accuracy of 93. 64% on average for the 11 class recognition. Although the SCRNN does not outperform the SLAYER based CNN network in 11 class classification, the SCRNN is still competitive at 90. 28%. We conclude this accuracy drop for the 11 class recognition task is due to the introduction of the additional class of random gesture. The "other" class in the DVS gesture dataset consists of random samples and each of those is neither same as other samples nor falls into the first ten categories. The SCRNN with designed recurrent convolution operation is found to be less effective to such type of training data. Although the SCRNN although does not outperform the SLAYER based CNN network in 11 class classification, the SCRNN is still competitive at 92.
CONCLUSION In this paper we presented a novel spiking convolutional recurrent neural network that was designed for efficient human hand gesture recognition. The individual cell is able to extract the spatial features by 3D spiking convolution operation and transferring the information recurrently. The SCRNN is successfully deployed to the DVS 128 gesture dataset. The SCRNN tested on the IBM DVS gesture dataset achieving an averaged recognition accuracy of 96. 59% for 10 category classification and 90. 28% for 11 category classification. We have shown that the designed SCRNN compared to standard feedforward CNN structure performs less competitive for the "unknown" class but has the advantages in terms of convergence speech and accuracy for the fixed amount of categories. However, we believe that the usage of SCRNN is not only limited to action recognition but can be extended to various dynamic scene recognition and prediction tasks.
Untargeted metabolomics using liquid chromatography-mass spectrometry (LC-MS) is currently the gold-standard technique to determine the full chemical diversity in biological samples. This approach still has many limitations, however; notably, the difficulty of estimating accurately the number of unique metabolites being profiled among the thousands of MS ion signals arising from chromatograms. Here, we describe a new workflow, MS-CleanR, based on the MS-DIAL/MS-FINDER suite, which tackles feature degeneracy and improves annotation rates. We show that implementation of MS-CleanR reduces the number of signals by nearly 80% while retaining 95% of unique metabolite features. Moreover, the annotation results from MS-FINDER can be ranked with respect to database chosen by the user, which improves identification accuracy.
Among the most advanced and versatile methods developed recently for untargeted metabolomics is the tandem MS-DIAL-MS-FINDER suite. MS-DIAL is an all-in-one program for metabolomics and lipidomics that relies on mass spectral libraries such as NIST 14 and MassBank of North America (MoNA) for metabolite annotation. MS-FINDER is a partner program of MS-DIAL, in which unknown structures can be elucidated from MS/MS spectra by the hydrogen rearrangement rulesbased scoring system. Here, we describe a third tool in this suite, called MS-CleanR, to remove degenerate features and improve annotation rates from untargeted LC-MS-based metabolomics data. Starting from the aligned peak list files determined by the MS-DIAL deconvolution process, our R package firstly removes noise signals by using generic filters.
In the second step, the package groups the ion features based on the results of the MS-DIAL peak character estimation algorithm  22  providing the ion linkages of adducts, correlated chromatograms, putative ion source fragments candidates and similar metabolite profiles among samples. In the third step, clustered ion features are merged between positive ionization (PI) and negative ionization (NI) modes and the adduct relationships are corrected accordingly. The cleaned-up feature list can be exported to MS-FINDER for annotation purposes. Finally, the package merges the MS-FINDER annotation output with the cleaned-up peak list and includes the possibility to prioritize identification according to the DBs used for MS-FINDER interrogation. The whole MS-CleanR workflow is easily accessible through a Shiny user interface (Figure  1 ) and it is available as open source code.
Mass detection was performed in positive ionization (PI) and negative ionization (NI) modes at 30 000 resolving power [full width at half maximum (FWHM) at 400 m/z] for MS1 and 17 500 for MS2 with an automatic gain control (AGC) target of 1e5. Ionization spray voltages were set to 3. 5 kV (for PI) and 2. 5 kV (for NI) and the capillary temperature was set to 256°C for both modes. The mass scanning range was m/z 70-1050 Da for standards and m/z 100-1500 Da for plant extracts. Each full MS scan was followed by data-dependent acquisition of MS/MS data for the six most intense ions.
Data processing LC-MS data were first processed with MS-DIAL version 4. 12. MS1 and MS2 tolerances were set to 0. 01 and 0. 05 Da, respectively, in centroid mode for each dataset. Peaks were aligned on a quality control (QC) reference file with a RT tolerance of 0. 1 min and a mass tolerance of 0. 015 Da. Minimum peak height was set to 70% below the observed total ion chromatogram (TIC) baseline for a blank injection. MS-DIAL data was cleaned with MS-CleanR by selecting all filters with a minimum blank ratio set to 0. 8, a maximum relative standard deviation (RSD) set to 30 and a relative mass defect (RMD) ranging from 50-3 000. The maximum mass difference for feature relationships detection was set to 0. 005 Da and maximum RT difference was set to 0. 025 min. The Pearson correlation links were considered only for biological datasets with correlation ≥0.
8 and statistically significant α = 0. 05. Two peaks were kept in each cluster: the most intense and the most connected. The kept features were annotated with MS-FINDER version 3. 26. The MS1 and MS2 tolerances were set to 5 and 15 ppm, respectively. Formula finder were exclusively processed with C, H, O, N, P and S atoms. DBs based on the genus and the family of the plant species (Table  S3 , Table  S4 , Table  S7 , Table  S8 ) being investigated were constituted with the dictionary of natural product (DNP-CRC press, DNP on DVD v. 28. 2) and the internal generic databases used were KNApSAcK, PlantCyc, HMDB, LipidMaps, NANPDB and UNPD. Annotation prioritization was done by ranking genus DB followed by Family DB and then generic DB (internal DB from MS-FINDER).
Statistical analysis Statistical analyses were done by using SIMCA-P+ (version 15. 0. 2, Umerics, Umea, Sweden). All data were scaled by unit variance (UV) scaling before multivariate analysis. The orthogonal projection to latent structure using discriminant analysis (OPLS-DA) was used to separate data according to A. thaliana growing conditions. The OPLS regression model used for the Medicago datasets was tuned with line resistance as the Y input: the following resistance indices 0, 1 and 2 were respectively indicated for the F83, A17 and DZA45. 5 strains. Coefficient scores were used to rank variables according to their class biomarker: a high coefficient indicating a strong correlation with resistance traits.
Mass spectral similarity network The. msp NI and metadata files generated at the end of the MS-CleanR workflow were imported into MetGem 25 (version 1. 2. 2). A mass spectral similarity network was created with a cosine score cut off fixed at 0. 65, a maximum of ten connections between nodes and at least four matched peaks. The resulting network was then imported into Cytoscape 26 (version 3. 7. 2) to tune visualization. Nodes were thus colored according to their annotated chemical classes and their sizes were indicated relative to the OPLS coefficient score. Edge width was deepened according to their cosine value.
025 minutes) can be computed, the strong correlation links being then considered during the clustering process. If the study involves the same set of samples acquired both in PI and NI mode, the MScombine 29 tool, incorporated into MS-CleanR, can be used to detect possible links between positive and negative features appearing in the same RT window. This process corrects misidentified relationships to consider observed m/z differences acquired between both ion modes. The package can only treat PI or NI data independently, however. We observed that a unique metabolite signal in each cluster can be selected by: a PI/NI adduct link (e. g.
[M+H] + /[M-H] -, [M+Na] + /[M+FA-H] -; the most intense peak of the cluster, and the peak with the most relationships to other features (i. e. the highest 'degree' of connection). Among each cluster, one to n features (tunable by the user) can be selected for further annotation: the most intense, the most connected or both. The other features are removed from consideration. Step 3: Feature annotation. After the above filtering steps, only a portion of the original features are exported to MS-FINDER, which greatly accelerate the processing time. This software computes feature annotations by querying internal DBs or imported DBs. Several DBs can be used to annotate a single set of features by exporting the results for each DB used. Additionally, a "compound level" column can be added into external DBs to further prioritize annotation within each DB used.
Workflow benchmarking on pooled standards To validate our approach, we benchmarked the MS-CleanR workflow by using a mixture of 51 NPs standards profiled in NI and PI modes with a reverse phase column and a 25 minutes gradient. The resulting data were compiled in an in-house DB comprising RT, HRMS and MS/MS fragmentation patterns (DB-level 1 annotation according to the Metabolomics Standards Initiative-MSI  32  ). To test whether the workflow retained features arising from unique metabolites and removed useless signals, we compared the results obtained by using a combination of MS-DIAL and MS-FINDER and DBlevel 1 annotation to those obtained by using MS-CleanR. For the latter, we created another DB of the same metabolite set encompassing accurate mass, molecular formula and SMILES strings (DB-level 2 annotation according to the MSI) to reproduce real-case annotation processing.
4). Formononetin displayed complex adduct relationships in PI and NI modes and successive features with higher mz's MS/MS fragment of formononetin in PI mode. The merging of PI and NI modes allowed the main feature in this complex cluster to be selected and provided a perfect match with level 1 annotation DB. The only mismatch was encountered for phloridzin due to the neutral loss of a glucose moiety in both in PI and NI modes. Only genine was detected in PI mode, resulting in selection of this signal in the final peak list. Insert Figure  3  To model more closely a real biological sample, we standardized our workflow by using a mixture of 167 standard compounds from the IROA Mass Spectrometry library (Table  S2 ). As above, we found significant feature inflation: 6732 signals after concatenation of PI and NI datasets (Figure  S2 ).
Evaluation of MS-CleanR on biological samples To evaluate the utility of the workflow on a real dataset, we set up an experiment to compare metabolome changes in Arabidopsis thaliana plants due to different culture conditions and age of the plants. Three cultural conditions were assessed (low density growth in Jiffy® pots for 32 days, high density growth in plastic pots for 21 days and hydroponic culture in liquid MS medium for 14 days) and 10 biological replicates were analyzed per culture condition. At harvest time, 4 leaves (2 cotyledons and 2 leaves) were observed for hydroponic plants, the densely seeding plants showed not more than two small, but completed, developed leaves, while the jiffy growing plants harbored large and well developed rosette leaves. Extracts were made from the aerial parts of the plants grown in pots and from the roots and green tissues of plants in hydroponic culture, and the extracts were profiled by LC-MS.
The datasets acquired in PI and NI modes were treated by using the MS-CleanR workflow with default parameters (see Methods). Sequential principal component analysis (PCA) was used to provide an unsupervised overview of the LC-MS fingerprints resulting from the generic filters and feature clustering (Figure  4 ). The PCA score plot of raw PI and NI mode data displayed 51% of total explained variance using the first two principal components. QC samples appeared in the center of the PCA score plot, demonstrating the reproducibility of the LC-MS analysis. As expected, the youngest plants growing hydroponically were completely separate on the first principal component (PC1) axis from the older plants growing in pots. The plants growing in Jiffy pots and plastic pots could not be distinguished in the raw dataset.
After the generic filter step, the data from these latter two conditions formed more distinct clusters, the total explained variance was slightly improved at 58% and the number of features decreased by 35% (Figure  4 ). After the feature clustering step, the number of features was reduced by 80%. All datasets were annotated with in-lab DB (level 1) and with MS-FINDER (level 2) by reference to external DBs of Arabidopsis (Table  S3 ) and Brassicaceae compounds (Table  S4 ) and an internal MS-FINDER plant-related DB (comprising PlantCyc, KNApSAcK, HMDB, LIPID MAPS and UNPD). In the raw PI and NI dataset exported from MS-DIAL (1163 features), 42% of all features were annotated, 26% of them appeared in the Arabidopsis DB, 2% in the Brassicaceae DB,7% in the internal MS-FINDER DBs and 6% with in-lab DB (Figure  4 ); 58% of all features were unidentified.
truncatula that have different levels of resistance to A. euteiches infection: strain DZA45. 5 has the highest level of resistance, A17 an intermediate level, and F83 is the most susceptible  36. These three strains were analyzed by LC-MS in NI mode and potential biomarkers were highlighted by multivariate data analysis (Table  S6 ). The metabolites that were differentially produced in the two most resistant strains (A17 and DZA45. 5) when compared to the more sensitive one (F83) were identified by OPLS regression. Insert Figure  5  After application of the MS-CleanR workflow, the PCA score plot showed a net clustering of the samples from each strain of M. truncatula. QC samples were centered on the PCA plot demonstrating very good reproducibility (Figure  5 ). When annotated by reference to DBs from Medicago or the legume family Fabaceae, 60% of the dataset was annotated (Figure  5 ) and an additional 9% with MS-FINDER DBs.