dataesr/scientific-paragraphs-categorization huggingface dataset:
III. RESULTS To be able to compare with state-of-the-art results and the performance indexes, we have applied the proposed method on three different datasets. The first dataset is the well known handwritten digit dataset (MNIST  [41] ) used for image processing and contains static images (without temporal sparsity). As the content of MNIST frames is almost binary pixels (black and white) with a high amount of sparsity (on average, around 84% of pixels in each frame are zero), we decided to include this dataset for benchmarking. The second dataset is autonomous steering dataset (PilotNet  [42] ) which contains a video stream recorded by an installed camera in-front of a car and a synchronous recording of the steering angle as the ground-truth label. This dataset reveals how our proposed algorithm can improve processing performance by exploiting temporal sparsity.
The third dataset is a set of hand gestures recorded by an event-based camera (DVS128 Gesture Dataset  [43] ). Eventbased cameras or Dynamic Vision Sensors (DVSs)  [44]   [45]  [45]  [46]  are bio-inspired vision sensors with independent asynchronous pixels which generate events when they detect a change in light intensity. Dynamic Vision Sensors generate and transmit information of the visual scene in a completely different manner than conventional frame-based cameras. Several advantages (like high-speed sensing, high dynamic range, and low power consumption) result in their growing use in neuromorphic engineering and low power mobile applications. The sparse event-based data provided by these sensors can be directly consumed by our network 5.
A. Handwritten Digit Recognition (MNIST) As the first experiment, we have applied the proposed method using the MNIST  [41]  dataset. For this experiment, 5 As it will be explained, we need to use a frame to event conversion for the other two datasets we used binary output spikes (±1) for all hidden layers. Initially, we have trained an ANN for the MNIST dataset using standard stochastic gradient descent. For conversion of the ANN network to SNN, the weights and biases are used in the equivalent SNN. To do the conversion we need to define the threshold (equivalent to quantization level) and also a method to convert input frames to spikes. We converted the input frames to spikes with V alue = +1 by sending the pixels with higher intensity earlier in time ("linear intensity to delay" coding)  [18].
Additionally, pixels with an intensity equal or less than '0. 2' (pixel values are normalized between '0' and '1') will not fire at all (the quantization threshold for input frame is '0. 2'). Using this method, each MNIST frame is converted to a spike train with the duration of '0. 8' TU (time Unit)  6. We have trained a 4-layer convolutional neural network  7. Table  I  shows the results of the proposed method for the MNIST dataset. (16C5 -2AP 2 -8C5 -2AP 2 -256F C[50%Dropout] - 10F C) "Avg Num Spikes" in Table  I , reports the average number of spikes per input frame per neuron in the network. To measure the accuracy of the MNIST network, we accumulated the number of output spikes (considering the sign of spikes) and the maximum number was assumed as the winner class. In Table  I , we also added the results from the other stateof-the-art works on supervised training of Spiking Neural Networks.
B. Autonomous Steering (PilotNet) PilotNet is a neural network introduced by NVIDIA  [42]  along with the dataset 9. The dataset contains video recording (10 frames per second) by a camera placed in front of the car as well as the corresponding steering angels for each frame of the video 10. To find out the amount of redundancy (especially temporal sparsity) in this dataset, we compressed the raw frames using "Motion JPEG 2000 lossless compression" algorithm which resulted in 18 times reduction in size. In our SNN, we exploit sparsity in all the neural layers (not only input layer) but sparsity in the input can result in sparsity in deeper layers. PilotNet is a 10-layer feed-forward convolutional neural network (CNN) with input size of 3 × 66 × 200 and configuration of 24C5-36C5-48C5-64C3-64C3-1164F C -100F C - 50F C -10F C -1F C.
9 https://github. com/lhzlhz/PilotNet 10 https://youtu. be/ N7nC-8YxzE Algorithm 3 Generating input events from video stream, every pixel is considered as one neuron accuracy. Fig. 4  shows the output of the PilotNet for the original CNN inference and the proposed SNN inference for part of the dataset. Fig. 5  shows the number of required MAC operations during inference of PilotNet in the original CNN and also number of synaptic update (updates in Algorithm 2) in the SNN for the first 2800 frames  11. For these frames, average number of operations in SNN is 3. 1Mops and it decreases to 1. 7 for the first 20,000 frames which is more than 16 times less than plain CNN inference. The current dataset has a very low frame rate (10 fps). To find out how are the numbers for higher frame- rates, we made a 15min recording with a 480fps camera  12 which resulted in 140 times reduction in number of operations compare to the plain CNN.
C. Hand gesture recognition (DVS128 Gesture Dataset) In the third experiment, we have evaluated the performance of our network by using the hand gesture dataset from IBM  [43]. This dataset is recorded with a DVS camera for a gesture classification task and includes data recorded from 29 subjects, each performing 11 hand gestures under 3 different lighting conditions. Each recording is a stream of events with an average duration of 72 seconds. Events from DVS contain a binary value (±1 as the sign of spike), which means an increase in light intensity results in a positive spike and decrease in light intensity results in a negative spike. In an ideal case (ignoring the analog circuit artifacts and noise), a DVS pixel behaves very similar to what is described in Algorithm 3 except the fact that there is no frame-period.
To use standard back-propagation of error and GPU accelerated training, we train our networks in Tensorflow with a frame-based method. However, for this experiment, input data is event-based. To use our frame-based platform for training, we used the fact that neuron states in our SNN network are equivalent to neuron states in the frame-based network (when ignoring the quantization error). To build a dataset for training, we take periodic snapshots from the state of neurons in our intermediate leaky layer. These snapshots are used as the input frames for our ANN training platform. The snapshot rate should be high enough to capture the temporal dynamics of the scene. In this experiment, we used the sampling period of 64 ms. Using higher snapshot rate results in more accurate SNN conversion but also slows down the training.
As in this dataset, the presentation of each class takes 6. 5 seconds on average, one may choose different inference time windows for classification. We perform each inference by resetting the state of all neurons and then presenting a part of the recording with the length of inference time window. For example, when the inference time window is 640ms, a recording of 6. 4 seconds is divided into 10 individual recordings and results in 10 separate classifications. For the experiments in this section, we have quantized all of the synaptic weights to 8 bits and stored them as integer numbers between -128 to +127. We applied several spiking thresholds to neurons of hidden layers to illustrate the tradeoff between accuracy and number of operations  15. The results are summarized in Fig. 6  where accuracy is reported with the inference time window of 64 ms.
IV. DISSCUSSION In this work, we presented a method to convert a synchronous ANN to an asynchronous SNN and efficiently exploit spatio-temporal sparsity. Even though our goal was not trying to mimic biological neural networks, we used the sparsity and asynchronous execution features of bio-inspired implementations. Our motivation was to offer a better implementation of synchronous ANN since a direct ANN implementation is not easily scalable and won't easily exploit the sparsity of the network for efficient power consumption. We have shown our proposed method reduces the number of operations dramatically for different datasets while its impact on accuracy is negligible. In this paper, we compared the number of operations in an ANN with the number of operations in the proposed SNN. However, the type of operations on these two platforms are not exactly similar.
ACKNOWLEDGMENT This work was partly funded by  EU H2020  grants  687299  " NEURAM3 " and 824164 "HERMES", and by  Spanish  grant from the  Ministry of Economy  and Competitivity  TEC2015-63884-C2-1-P (COGNET)  (with support from the  European Regional Development Fund). The authors would like to thank all  GrAI Matter Labs  employees, specially  Ingolf Held ,  Gokturk Cinserin ,  Peng Qiao ,  Louis Rouillard  and  Georgi Valkov  for their kind support and interesting discussions. The authors would like to thanks the reviewers for the valuable comments and their contribution in improvement of this paper.
Statistical analysis Descriptive analysis, statistical tests and graphs were performed using the GraphPad Prism 7 (GraphPad, CA, USA), Past (3. 2) and Excel software (Microsoft). Kruskal-Wallis tests were realized to compare biomarkers between all groups, and the results were considered significant for p < 0. 05. Comparison of data between groups was performed by the Mann-Whitney U test for quantitative variables. Tree diagram was conceived in Smartdraw (https://www. smartdraw. com/).
Solving for the static deformation The equilibrium equations need to be transformed into a weak form in order to be implemented in the finite element software FreeFem++. We define the test functions v and w which satisfy v(1) = 0 and w(0) = w(1) = 0, and a test scalar B. By multiplying the equilibrium equations (  40 ), (  41 ) and (  34 ) by the test functions v, w, and B, integrating by parts and using the boundary conditions  (33) , a residual is defined: ([r, z, β], [v, w, B]) = 1 0 v H( R) R s1 z λ 1 d R - 1 0 βrr vd R + 1 0 w H( R) R s1 r λ 1 d R + 1 0 βrz wd R + 1 0 H( R) s2 wd R + (β atm + β)B Vs + 2π 1 0 rzr d R γ -(β atm + β pres )B( Vs + Vini ) γ. ( 42 ) The static equilibrium equation then simply reads: ([r, z, β], [v, w, B]) = 0 , ∀[v, w, B]. ( 43 ) This equation is solved using Newton-Raphson's method.
First an initial guess of the solution is made in the form of a spherical cap. Then the residual ([r, z, β], [v, w, B]) is linearized to provide the gradient for Newton-Raphson's method. Introducing the variables r = r0 + x r , z = z0 + x z and β = β 0 + x β , the linearized weak form reads: ([r 0 , z0 , β 0 ], [v, w, B]) + ([r 0 , z0 , β 0 ], [v, w, B]) , [x r , x z , x β ] = 0. (44) The explicit linearized form of Eq. (  44 ) is provided in Appendix A. The linear equation (  44 ) is solved at each step of Newton-Raphson's method to find the increment x = [x r , x z , x β ]. The initial guess is then updated: x0 = [r 0 , z0 , B 0 ] = x0 + x. This is repeated until x is sufficiently small ( x 2 < 1e -8 ). FreeFem++ software  [29]  is used to assemble the stiffness matrix and the force vector, and to solve the non-linear inflation problem.
SVM and RF can handle data sets with large numbers of features, such as the various spectral bands for the different dates of satellite image time series, and with many training samples covering a wide geographical extent. In addition to image time series, high-quality labelled reference data are an essential element for an efficient classification  [11]. Reference data can come from various sources such as dedicated field surveys, or existing databases. Field surveys are expensive and tedious to perform since it is necessary to visit the fields several times a year to handle the seasonal land-cover classes such as crops. Therefore, reference data are generally available after the corresponding mapping period, whereas the remote-sensing images are available a few days after their acquisition.
In essence, the delays in obtaining reference data are the main obstacles to the timely frequent production of land-cover maps. These delays are even longer when using publicly available databases since those have their own production and updating cycles. Naturally, the delays are proportional to the area covered and the kind of classes considered. Consequently, the large scale (i. e. , over large areas) land-cover mapping  [12]  at the annual frequency cannot rely on field surveys nor existing data bases. In this study, we propose and evaluate methods to overcome the absence of reference data for the period to be mapped. The purpose is to classify an image time series by using prior information as past image time series and past reference data. However, the use of previous data is not straightforward due to changes between years.
The problem of classifying a large image with a small area covered by reference data is not addressed in this work, as in a current use the images are split according to eco-climatic areas before mapping  [12]  and the reference data are an excellent representation of the mapped area. In this work, the Formosat-2 images are included in a unique eco-climatic area. A split into eco-climatic areas seems to be more efficient than this use of DA in our case, as a complete satellite image can cover very different landscapes, including seaside mountains and plains for example, and an essential predicate for the DA is that the two domains must be similar. Moreover, removing features is not considered at the moment as the RF chooses relevant features during the training phase. Coupled with a scattered reference data on the image, the RF has a great generalization capability  [20].
Image Time Series A large data set composed of 7 years of Formosat-2 images was used for this study. Formosat-2 has a one-day revisit cycle and a spatial resolution of 8m. The images were first atmospherically corrected using MACCS  [28]. Masks for clouds, cloud shadows, and saturated pixels were provided as a by-product of the correction. Using the four spectral bands: blue, red, green, near infrared (NIR), two spectral indices were computed: NDVI (Normalized Difference Vegetation Index) and brightness (the norm of the spectral vector). As shown in Table  1 , the number of available images and their repartition are different for each year, and therefore, the time series used as features for the classification have different lengths. To have the same number of features for each year, each time series is resampled onto a regular time grid.
Reference Data Reference data for the seven satellite image time series were obtained by field surveys. These field surveys are designed to visit a high number of diverse crop plots while minimizing the travel distance. Each annual crop plot is visited each year. This may not be the case for perennial crops, which can be assessed by photo-interpretation. The resulting reference data is a stratified opportunistic sampling, which is not exactly proportional to the distribution of classes in the entire study area, although ranks are preserved. Figure  1  is an illustration of a true-color Formosat-2 image and the polygons resulting of the corresponding field campaign for 2010. The reference data covers large and diverse parts of study area. These data can be used for classifier training and validation purposes.
Validation The validation of the DA techniques presented in this paper will be carried out classically since reference data are available for all image time series used in this work. The validation data will be used to compute confusion matrices and derived accuracy metrics. In the confusion matrix, the rows correspond to classes in the reference data, and the columns to the labels produced by the classifier. Each cell in the matrix indicates the number of samples belonging to the class given by the reference and classified as belonging to the class indicated by the column. The diagonal of the matrix represents the number of correctly classified pixels. The Overall Accuracy (OA), is the total number of correctly classified pixels divided by the total number of validation pixels. The FScore is the harmonic mean of precision and recall: f score = 2 × recall × precision recall + precision where: • precision is the ratio between the correctly classified pixels and the sum of all pixels classified as this class; • recall is the ratio between the correctly classified pixels and the total number of reference data pixels of that class.
Sampling Selection Strategies In a real case of use, several elements are available for each domain. The source domain is the most complete, with the time series, the reference data, the trained model and the supervised classification. For the target domain, the time series and naive classification are available. Four sample selection methods are proposed, using all these elements: 1. The source domain reference data provides the pixel positions. Then they are used to select samples in both domains: (a) The selected samples are used to estimate the EMD transport. This approach is called SP-EMD.
The use of voting methods allows several source domains to be jointly used and therefore benefits from the gain due to transport to improve the performance of voting methods. Figures  8  and 9  show that this gain remains minimal given the workload involved in the effective use of OT. Again, the performances are boosted by the combined use of PAS and EMD-PerClass, allowing an improvement in OA near to 10% in comparison to the naive-based voting. This approach provides a better OA with mostly correctly classified pixels and fewer undecided and incorrectly classified pixels. The results of this study provide a negative overview of the use of OT, which is mainly due to the quality of the dataset, and its strong heterogeneity. The 2008 case is significant because the absence of images for a long duration (see Table  1 ) makes the use of this year as a source domain difficult.
More precisely, we have identified the following limitations from analysing several case studies and threat models: (a) Spurious attacks when honest authorities are present in different phases of the voting process. For many threat models, this excludes modelling a registrar that distributes credentials in a registration phase and then commits credentials of eligible voters to the ballot box in a later phase, as in JCJ  [22]  and Belenios  [23]. (b) Spurious attacks with ProVerif when ballot secrecy notably relies on the freshness of some data coming from previous phases. For example, such data can be credentials created during a registration phase, as in JCJ and Belenios. (c) Spurious attacks for threat models in which no dishonest voter is assumed (we will explain later why this is a more complex case than with dishonest voters that we handle as well).
The swapping approach also suffers from systematic precision issues when the security relies on the freshness of data created in previous phases. Indeed, the compiler introduces many new internal communications in the produced biprocesses. However, the very abstract treatment of internal communication used by ProVerif causes the tool to also explore the possibility of swapping data with an old session whose data has already been swapped before. We consider this to be a significant limitation, which manifests itself as spurious attacks e. g. the ones for JCJ and Belenios (see Figure  3 ); the credential being the fresh data coming from the registration phase and used during the voting phase. We provide more details in  [27]. b) Small-attack property: A different line of work is to devise small attack properties, as for example in  [17]  for ballot secrecy.
VIII. CONCLUSION We presented three conditions that together imply ballot secrecy. They proved to be tight enough to be conclusive on several case studies. Verifying ballot secrecy in a modular way via our conditions constitutes a new approach which outperforms prior works: we cover a greater class of e-voting protocols and threat models, and the analysis is more efficient. Our new approach has also opened several avenues for future work. First, our notion of tally is currently limited. Hence, our method is currently unable to deal with revotes. While adding revotes might be directly achievable, we conjecture that considering revoting policies (e. g. ballot weeding in Helios as analysed in  [17] ) is a more intricate challenge. Furthermore, our notion of tally cannot deal with homomorphic tallying and only produces a set of votes, while e-voting protocols satisfying verifiability should also produce verification data (e.
2016). Of the 1134 strains analyzed, 884 were sequenced in this study and 250 were from public databases (for details, see Supplemental Tables  S1, S2 , respectively). The strains were isolated in various geographic regions (21 countries, six continents) (Supplemental Fig. S1 ). A total of 227 (∼40%) of the bovine commensal isolates and four of the human commensal isolates were stx and/or LEE positive (Table  1 ). We first constructed a neighbor-joining (NJ) tree based on seven housekeeping genes of the 937 commensal strains and 34 completely sequenced E. coli reference strains (Fig. 1A ) with E. fergusonii and cryptic Escherichia clade 1 strains as outgroups. The NJ tree clearly showed that most bovine commensal strains (549/575) belonged to a large monophyletic lineage (referred to as bovine-associated lineage) that is distinct from a human strain-dominated lineage (referred to as human-associated lineage).
2013). Most bovine (96%; 552 out of 575 strains) and human (73%; 264 out of 362 strains) commensal isolates were grouped into the bovine-and human-associated lineages, respectively, although some portions of human commensal isolates were in the bovine-associated lineage, especially phylogroup A strains. We obtained almost same distribution patterns in the randomization test analyzing equal numbers of bovine and human commensal strains (300 strains randomly selected from each group) (Supplemental Table  S3 ). Although Japanese isolates were most prevalent in the strain set, strains isolated in various geographic regions were diversely distributed among the Japanese isolates in the core gene tree. In addition, among the 196 O and 53 H serogroups in the E. coli O and H antigen gene database (EcOH database)  (Ingle et al.
These workers are required by law to undergo periodic fecal examination. Animal handlers were not included in the examinees. For the isolation of human commensal E. coli, only one colony was picked from a XM-G plate for each stool sample. Furthermore, we collected 103 E. coli strains (ExPEC strains) isolated from blood or urine specimens of patients from various hospitals in Japan. In total, 1149 strains were sequenced in this study. Genomic DNA was purified from 1 mL of an overnight culture of each strain using a DNeasy blood and tissue kit (Qiagen). Genomic DNA libraries were prepared using the Nextera XT DNA sample preparation kit (Illumina) and sequenced using the Illumina MiSeq platform to generate 300-bp paired-end reads. The genome sequence information of 52 bovine commensal and 86 human commensal strains and 112 clinical strains (STEC, EPEC, and ExPEC) isolated in various countries was obtained from public databases along with metadata (Supplemental Table  S2 ).
All genome data of bovine and human commensal isolates available in the NCBI database (accessed in January 2019) were included in this study. As for clinical STEC and EPEC strains, we first selected the strains, for which necessary metadata including country, host, and clinical significance (disease or symptom) are available, from those in the NCBI database (accessed in August 2017), and then one strain was randomly selected from each serotype (according to the original description). Regarding ExPEC strains, we selected strains with the metadata mentioned above from the strains in three BioProjects (PRJNA269984, PRJEB9927, PRJNA266030) and further selected 26 strains having different serotypes and not closely related to each other (less than five SNP distance).