dataesr/scientific-paragraphs-categorization huggingface dataset:
Materials and Methods Germplasm Collection. The germplasm studied here was comprised of a total of 10,038 pepper accessions. Based on genebank passport information, these accessions originated from 130 countries across five continents and represented 9,689 accessions from the domesticated species, 83 from seven wild species, and 266 (2. 5%) had an undetermined species status. Details of species, genebanks, and passport information provided by provider curators are reported in SI Appendix, Table  S1  and Dataset S1. DNA Isolation and Library Construction. Genomic DNA was extracted from 100 mg fresh leaf tissue collected from individual plants for each accession. DNA extraction was performed either with the DNeasy Plant Mini Kit (QIAGEN) or the Sbeadex maxi plant kit (LGC Genomics). DNA quantity and quality parameters were assessed using both spectrometry (ND-1000; NanoDrop, ThermoScientific) and fluorometry (Qubit 2.
GWA Analysis. Phenotypic data for 21 qualitative/pseudoqualitative descriptors for plant, flower, and fruit trait categories recorded by each genebank during multiplication cycles following the International Plant Genetic Resources Institute (IPGRI)/Bioversity (57) protocol for the genus, were analyzed, including three morphological plant traits, six flower, and 12 fruit traits (SI Appendix, Table  S9 ). Before proceeding with the analysis, all data have been reviewed removing any inconsistencies (e. g. , traits not registered equally) and outliers. This led to the establishment of a core set of 2,059 C. annuum accessions representing the variation of the whole species gene pool (SI Appendix, Fig. S14 ) and without any missing data for 21 phenotypic observations (Dataset S4). In order to widen the genetic variants detected in GWAS such as major rearrangements, insertions, and deletions, reference-free GWA analysis was implemented using a k-mer GWAS pipeline  (58).
This pipeline uses k-mers, as markers for GWA analysis. Firstly, the 31-bp k-mer sequences were identified from the quality trimmed reads from each sample using KMC (version 3. 0). The software was run with minimum read count threshold of 2 and keeping other parameters to default. The k-mers from all the samples were then compared to generate a k-mer presence/absence matrix which was further Global range expansion history of pepper (Capsicum spp. ) revealed by over 10,000 genebank accessions filtered with minor allele frequency (MAF) ≥ 0. 02 and minor allele content (MAC) ≥40. The kinship matrices of relatedness between the samples were calculated based on the filtered k-mer matrix and using EMMA package  (59)  with default parameters. Finally, GWAS was carried out using mixed linear model (MLM) from package GEMMA (version 0.
96)  (60)  for each trait separately. The P value threshold was determined based on permutations of phenotype. For visualization of the results, the top 1 million k-mers for each trait were extracted from GWA study and mapped against the reference genome sequence C. annuum CM334 version 1. 6, and positions of uniquely mapped k-mers were retrieved. The Manhattan plots for each trait were generated using qqman R package  (61). Data Availability. All study data are included in the article and/or supporting information. The raw data sequences (FASTQ files) are available on the European Nucleotide Archive (ENA) under the accession number BioProject PRJEB45375. The source codes used to perform ReMIXTURE analysis (including visualization scripts) are available on GitHub (https://github. com/ mtrw/ReMIXTURE).
ACKNOWLEDGMENTS. We gratefully acknowledge  Annette Marlow ,  Beate Kamm ,  Jacqueline Pohl ,  Susanne König ,  Ines Walde ,  Mary Ziems , and  Anne Massire  for technical assistance. We wish to acknowledge the  G2P - SOL  project, funded by the  European Union   Horizon 2020 research and innovation program  under Grant Agreement No. 677379.
Computer network systems are often subject to several types of attacks. For example, an excessive traffic load sent to a web server for making it unusable is the main technique introduced by the Distributed Denial of Service (DDoS) attack. A well-known method for detecting attacks consists in analyzing the sequence of source IP addresses for detecting possible anomalies. With the aim of predicting the next IP address, the Probability Density Function of the IP address sequence is estimated. Anomalous requests are detected via predicting source's IP addresses in future accesses to the server. Thus, when an access to the server occurs, the server accepts only the requests from the predicted IP addresses and it blocks all the others. The approaches used to estimate the Probability Density Function of IP addresses range from the sequence of IP addresses seen previously and stored in a database to address clustering, for instance via the K-Means algorithm.
Instead, the sequence of IP addresses is considered as a numerical sequence in this paper, and non-linear analysis of this numerical sequence is applied. In particular, we exploited non-linear analysis based on Volterra Kernels and Hammerstein models. The experiments carried out with datasets of source IP address sequences show that the prediction errors obtained with Hammerstein models are smaller than those obtained both with the Volterra Kernels and with the sequence clustering based on the K-Means algorithm.
History-based IP filtering One of the first work on the field of IP filtering is  [24]. An algorithm called History based IP Filtering (HIF) is proposed in their work. During an attack, HIF allows the access to the website only to the IP addresses belonging to a database containing all frequently observed addresses (IAD). The idea behind this algorithm was introduced in  [20]. In this paper, it is observed that the IP addresses from Code Red worm attacks were different from the standard IPs accessing the website. Thus, HIF adds an IP address to the IAD only if a certain threshold is exceeded. This threshold can be measured with different methods, e. g. , a certain number of packets, a fixed number of visits in a certain time window, etc. The advantage of the HIF algorithm is the low computational load required for its implementation.
Clustering of source address prefixes In  [22] , the authors introduce algorithms for mitigating DDoS attacks by filtering source address prefixes. Unlike AHIF, different sizes network are considered. The authors use a (1) (s) = n s ∑ N-1 i=0 n s i (2) r = reject if (s) ≥ accept otherwise hierarchical agglomerative clustering algorithm with single linkage for finding the most appropriate network masks. The used distance measure is defined with respect to network boundaries. In general, both the amount of requests from a source network and the neighboring density estimation (from a generalized clustering method) are taken into account. Unfortunately, the proposed technique is not applicable in practice on large source IP datasets since hierarchical clustering methods consume a lot of memory, (see  [29] ).
Experiments In order to prove the effectiveness of the proposed approach, in this Section we present our experimental results for a simulated dataset and a real one. In this Section, a comparison of the two approaches (namely the prediction based on Volterra systems and the one based on the Hammerstein one) is presented. The following method has been used to generate the simulated dataset: 1. generate a set of N = 1000 IP addresses in which the 10% are malicious users. For each of them a normal random variable generates the number of access. In particular the distribution chosen is N( , ∕5) where = U  [10, 100]  if the IP address is not malicious, = U[200, 1000] if the user is malicious; 2. during each time step t: -pick a random number using a Bernoulli with probability p = 0. 001 , i. e.
g. , HTTP/1. 0); the remaining 6 bits indicate the response status code (e. g. , 200 OK). type if the request required a file, it is the type of file. It is generally the file extension (e. g. ,. html), or the presence of a parameter list. server indicates the IP address of the server which handled the request. The upper 3 bits indicate which region the server was at; the remaining bits indicate which server at the site handled the request. The dataset collects data about the access for 87 days. We split the data in two parts: the first one is used for initializing the estimator in Eq. (37), while the second one is used as test set by using a rolling horizon method (as in  [7] ). Particularly, for each day t we compute the estimation by using all the IP observations in the previous days [0, t -1].
Further remarks: Integration with big data analytics systems Recent studies, such as  [28] , have clearly shown the strong requirement of integrating cyber-security frameworks, like the one we presented on our paper, with innovative big data analytics systems. Our technique, being intrinsically a big data analytics technique, is immediately prone to adhere to this nice integration. Indeed, when the input dataset to be processed become excessive large, i. e. it takes the form of a big data repository, scalability becomes a critical requirement towards the reliability of the overall cyber-security framework. This way, cyber-security techniques must be seamlessly integrated with state-of-the-art big data processing platforms, such as Hadoop, Spark and so forth, in order to obtain truly-scalable analytics systems over big data repositories (just like collections of IP addresses deriving from a portion of the Web).
Second, we think that the exploitation of knowledge management methodologies can be beneficial for the proposed algorithms (e. g. ,  [6, 9, 15] ). Funding Open access funding provided by Università della Calabria within the CRUI-CARE Agreement. This research has been partially supported by the French PIA project "Lorraine Université d'Excellence", reference ANR-15-IDEX-04-LUE. Open Access This article is licensed under a Creative Commons Attribution 4. 0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material.
This property is expressed by the assertion at line 8. Example 1 abuses pointers to bypass the C type system, a common practice in low-level programming known as type punning. Alternatively, some implementations rely on bitwise arithmetics. E. g. , if x and y have type uint32_t, the little-endian case may be rewritten as ((x & 0xff000000) >> 24) | ((x & 0xff0000) >> 8) | ((x & 0xff00) << 8) | ((x & 0xff)  << 24). Other implementations rely on compiler built-in functions, or assembly code, possibly using dedicated processor instructions. Examples can be seen in the Linux implementations of the POSIX htons and htonl functions, converting values between host and network byte-order. Our analysis is able to analyze all the above C implementations successfully, as well as alternative implementations (with stubs for assembly code).
Related work. Several approaches to endian portability are developed in the literature. [31]  relies on a source-to-source translation, which is only sound with respect to annotations provided by the programmer, whereas we require no annotations. [6]  extends a compiler to generate code that executes with the opposite byte order semantics as the underlying architecture, at the cost of a performance penalty. Annotations are also required for soundness in some cases. [22]  relies on dynamic analysis, which can find portability errors, but cannot prove endian portability formally, unlike our method. The Sparse  [7]  static analysis tool used by Linux kernel developers relies on pervasive type annotations to detect endiannesses issues, but comes with no formal guarantee. To our knowledge, no prior work uses sound static analysis to infer endian portability.
Contributions. The main contributions of this work are: -We present a novel concrete collecting semantics, relating the behaviors of two versions of a program, running on platforms with different endiannesses. -We propose a joint memory abstraction able to infer equivalence relations between little-and big-endian memories. -We introduce a novel symbolic predicate domain to infer relations between individual bytes of the variables in the two programs, which has near-linear cost, and the right amount of relationality to express (bitwise) arithmetic properties relevant to endian portability. -We implemented our analysis on the Mopsa  [30, 21]  platform. Our prototype is able to scale to large real-world industrial software, with zero false alarms. The paper is organised as follows. Section 2 formalizes the concrete collecting semantics, Sect.
Evaluation We implemented our analysis into the Mopsa platform  [30, 21]  designed to support modular developments of precise static analyses for multiple languages and multiple properties. Our prototype is composed of 3,000 lines of OCaml: 45% for the memory abstraction, 36% for the symbolic predicate domain, and 19% for double program management and iterators. It leverages 31,000 lines (excluding parsers) of elementary functions of Mopsa: framework and utilities (64%), generic iterators and numeric domains for analyses of all languages (11%), specific iterators and memory domains for the C language (25%). We have experimented our prototype on small idiomatic examples, open source software, and large industrial software. The analyses were run on a 3. 4 GHz Intel® Xeon® CPU.
Idiomatic examples We first check the precision and robustness of our analysis against a collection of small double C programs (between 20 and 100 LOC), inspired by various implementations of byte-swaps in Linux drivers, POSIX htonl functions, and industrial software. A set of 9 programs illustrate network data processing. These programs are similar to Example 1 of Sect. 1. They receive an integer from the network, increment it, and send over the result. Necessary byte-swaps are implemented for little-endian versions of these programs. Each example program implements a different byte-swapping technique on a 2, 4, or 8-byte integer: type-punning with pointer casts (like in Example 1), unions, or bitwise arithmetics. Refer to Examples 4, 5, and 6 in artifact  [17]  for the source codes. We also analyze Example 2 from Sect.
Open source benchmarks We then check the soundness, precision, and modularity of our analysis on three benchmarks based on open source software available on GitHub, with multiple commits for bug-fixes related to endianness portability. Refer to Examples 10, 11, and 12 in artifact  [17]  for relevant source codes excerpts. We analyze slices between 100 and 250 LOC, using primitives assume_sync and assert_sync for modular specifications of program parts. Our first benchmark is an implementation of a tunneling driver  [32]  based on the Geneve  [19]  encapsulation network protocol, which uses big-endian integers as tunnel identifiers. The driver was introduced in the Linux kernel, and patched several times for endianness-related issues detected by Sparse  [7]. Then, a performance optimization introduced a new endianness portability bug, which Sparse failed to detect.
It was fixed a year later. Our analysis soundly reports this bug, as well as previous issues detected by Sparse. It reports no alarm on the fixed code. Our second benchmark is a core library of the mlx5 Linux driver  [24]  for ethernet and RDMA net devices  [23]. We analyze a slice related to a patch, committed to fix an endianness bug introduced 3 years earlier, and undetected by Sparse despite the use of relevant annotations. The fix turned out to be incomplete, and was updated 6 months later. Our analysis soundly reports bugs on the two first versions, and no alarm on the third. Our third benchmark is extracted from a version of Squashfs [35], a compressed read-only filesystem for Linux, included in the LineageOS  [34]  alternative Android distribution. We analyze a slice related to a patch, committed to fix an endianness bug undetected by Sparse due to a lack of type annotations.
g. assisted surgery, autonomous vehicles. ). More generally, deep learning systems are often deployed as black boxes trained on huge available datasets, and are therefore lacking understandability and interpretability: reducing the number of parameters can help in visualizing and explaining decisions. Additionally, datacenters are increasingly used for deep learning, and their impact on the environment is becoming a concern. For all these reasons, there have been a significant amount of works aiming at reducing the size and computational cost of CNNs. To cite a few, some works propose to prune the connections in deep learning systems so as to reduce their numbers  [1]. Other works propose to distillate the knowledge from large architecture to smaller ones  [2]. It is also possible to focus on reducing the bit precision of weights and/or activations  [3]  or to factorize some of the operations  [4] ,  [5].
In this work, we propose to introduce a new factorized deep learning model, in which the factorization is not learned during training, but rather imposed at the creation of the model. We call these models ThriftyNets, as they typically contain a very constrained number of parameters, while achieving top-tier results on standard classification vision datasets. The core idea we introduce consists in recycling the same convolution to be applied a large number of times when processing an input element. In more details, the main claims of our paper are: 1) We introduce ThriftyNets, a new family of deep learning models that are designed with a fixed number of parameters and variable depths and flops.
2) We perform experiments on standard vision datasets and show that we are able to outperform current deep learning solutions for tiny parameters budget. 3) We design experiments to stress the impact of the hyperparameters of the proposed models on the accuracy and the flops of obtained solutions. The outline of the paper is as follows. In Section II we present related work and discuss the context of our proposed method. In Section III we introduce the proposed methodology and discuss the role of hyperparameters on the total number of parameters. In Section IV we perform experiments using standard vision datasets and compare the proposed method with existing alternative architectures. Section V is a conclusion.
Distilling a model into itself, or self-distillation, has also proven to be effective when iterated  [23]. While individual knowledge distillation focused on the student mimicking the outputs of the teacher, relational knowledge distillation  [24] ,  [25]  made it reproduce the same relations and distances between training examples, yielding a better representation of the latent space for the student, and better generalization capabilities. d) Efficient scaling: While the diversity of datasets in computer vision does not cease to increase, some works are focused on the re-usability of architectures that perform well on specific tasks. Mainly, the resolution and complexity of the input image play a great role on the minimal number of parameters required to achieve good performance. While it is possible to scale architectures by adding layers (depth increase)  [26]  or by adding filters (width increase)  [6] ,  [27] , a correct balance between those hyperparameters has been shown to lead to better results  [8].
B. Thrifty Networks Consider a problem in which the aim is to associate an input tensor x with an output y through the network function f. This network function is trained using a variant of the stochastic gradient descent algorithm to minimize a loss function L over a dataset D. We propose to define a convolutional layer C, parametrized by weights W, and to build a deep neural network using only this layer iteratively applied T times on successive latent representations of the input. Note that we do not use a convolution with bias in this work. This architecture, called ThriftyNet, is then defined by the following recursive sequence:    x 0 = PAD(x) x t+1 = D t [BN t (x + σ(W x t ))] y = FC(x T ) , (1) where PAD creates extra channels filled with constant values to extend the dimension of x, BN is a batchnorm layer, D t is a downsampling operation (typically achieved with strides or pooling) or the identity function, and FC is a final fully connected layer.
IV. EXPERIMENTS In this section, we explore the performances obtained with various values of hyperparameters, namely the total number of iterations T , the number of downsamplings performed, the history, and the number of filters f , which is directly linked to the number of parameters. Experiments are performed on CIFAR-10, CIFAR-100 and SVHN. CIFAR-10 and CIFAR-100  [36]  are datasets of tiny colored images of 32x32 pixels. They contain both 50,000 samples for training and 10,000 samples for test. CIFAR-10 is made of 10 classes, whereas CIFAR-100 contains 100 classes. Note that state-of-the-art performance is 98. 9% on CIFAR-10 and 91. 70% on CIFAR-100 using EfficientNet-B7  [8]  (64M parameters). SVHN is a dataset of digit classification from pictures of house numbers. Images are also of size 32x32 and present up to three digits in them.
A. Impact of data augmentation As we work with tiny networks, regularization techniques like heavy data augmentation, mixup or cutmix, designed to help very expressive networks to achieve better generalization performance, have little to no effect, as observed in our experiments. In Table  II , we report the performance on both the train and test sets we obtained when using a residual ThriftyNet with 40k parameters and 20 iterations. We observe very little impact in using more advanced techniques of data augmentation. We hypothesize that thrifty architectures are less likely to cause overfit because of the very constrained number of parameters. On SVHN, a 20K parameter ThriftyNet, trained with Auto Augment  [37]  and Cutout of size 8  [38]  achieves 97,25% accuracy, compared to the 96,59% of the same architecture trained on the raw dataset.
B. Comparison with standard architectures We then compare in Table  III  and in Table IV the performance of the proposed ThriftyNet architectures to standard ones found in the literature, resp. on CIFAR-10 and CIFAR-100. When available, the reported scores are those obtained with standard data augmentation. Otherwise, they are obtained using only the raw training set. The first observation that we can draw from Table  III  and Table  IV  is that ThriftyNets present competitive results with the literature for a tiny parameter budget. The Residual ThriftyNet we use on CIFAR-10 achieves up to 91% accuracy while using less than 40K parameters in total. The only size-comparable architecture we found is the one introduced in  [35] , but its accuracy is significantly lower than that of the ThriftyNets.
F. Effect of the number of filters Next, we investigate the evolution in accuracy when the number of filters f increases. Of course, increasing the number of filters leads to more parameters, as shown in Table  I. Figure  5  shows the evolution of accuracy for CIFAR-10 and SVHN in function of the number of parameters, obtained when varying f. In both cases, we observe that the trade-off between number of parameters and accuracy is not linear: reaching 1% extra accuracy can be very costly in terms of the required number of parameters, if the accuracy is already high. This trade-off is even sharper in the case of SVHN where we observe a two-step phenomenon, where accuracy is first increased quickly but then saturates. We think that this is a consequence of the difficulty of considered datasets: achieving 93% accuracy on CIFAR-10 is significantly harder than in the case of SVHN.