dataesr/scientific-paragraphs-categorization huggingface dataset:
Genome Collection: Phenotyping and Genotyping Assays The genome sequence files from 100 Enterococcus faecium and 97 Enterococcus faecalis strains that were previously obtained  (Tyson et al. , 2018a,b) , and whose AMR profiles were further characterized employing a panel of 9 antibiotic drugs  (Tyson et al. , 2018b) , were downloaded from the NCBI repository (Accession: PRJNA292665 and PRJNA292669) in July 2019, and alternatively, have been made available from a dedicated Zenodo repository  (Panunzi, 2019). The main biological and genomic features, including their genome, assembly and BioSample Accession IDs, sequence coverage, N50 values as well as isolation sources and antibiotic drug susceptibility characterization can be found in Table  S1. The authors  (Tyson et al. , 2018b)  appraised the antimicrobial susceptibilities of the selected strains by comparing the minimum inhibitory concentration (MIC) values, which were determined using an automated broth microdilution method (Trek Diagnostics, Independence, OH) with the US National Antimicrobial Resistance Monitoring System (NARMS) CMV3AGPF plates, to the breakpoints defined in the 2015 Clinical and Laboratory Standards Institute (CLSI) guidelines  (CLSI, 2015).
However, since the analyzed dataset is mainly composed of bacterial isolates derived from animals, the zone diameter interpretations should be made using veterinary standards. Regrettably, this relevant information was not provided by  Tyson et al. (2018b). Therefore, in our present study we specify that these reported susceptibilities were employed only for validation purposes. Afterwards, the contrast between the measured and the MIC breakpoint values allowed the determination of the "resistant" and "susceptible" phenotypes. With reference to the ARG determination at genomic level,  Tyson et al. (2018b)  employed the ResFinder  (Zankari et al. , 2012)  and the NCBI Pathogen Detection (https://www. ncbi. nlm. nih. gov/pathogens/) DBs. Thereafter,  Tyson et al. (2018b)  added two further categories-"resistant*" and "susceptible*"-when differences between genotype (presence or absence of corresponding ARG) and phenotype (resistant or susceptible) occurred.
Genome Annotation, Pan-Genome Estimation, and Phylogeny Assessment The automated annotation of genomes from both datasets (Accession: PRJNA292665 and PRJNA292669) was performed using PROKKA v1. 11  (Seemann, 2014) , while the core-genome and pan-genome were estimated using CD-HIT  (Fu et al. , 2012)  by clustering predicted genes when ortholog loci shared ≥ 95% of amino acid sequence identity and ≥ 95% of alignment coverage. A total of 424 genes existing in all strains were estimated to compose the core set. Afterwards, those 424 genes were aligned using MUSCLE  (Edgar, 2004a,b)  and concatenated into a final MSA file comprising 327,376 bps of DNA sequence. A phylogenetic tree was built from this core-genome MSA after filtering recombination regions using GUBBINS  (Croucher et al. , 2014)  under default parameters.
Phylogeny and Distribution of AMR Phenotypes A total of 100 E. faecium and 97 E. faecalis isolates constituted our validation set. This collection of genomes had been sequenced and phenotyped-phenotyping is described in Methods, while genome samples accession codes and phenotyping data are included in Table  S1 -in previous studies  (Tyson et al. , 2018a,b). However, the authors did not include a phylogeny of the analyzed samples. In our present study, the annotation of the 197 genome assemblies produced a sum of 531,915 putative protein-coding sequences, that were subsequently grouped into 13,702 clusters of orthologous genes (COGs). The pan-genome calculation evinced that 424 of these COGs were present in all the samples and constituted the core-genome. The successive alignment and catenation of core-genome COGs procured a MSA composed of 327,376 nucleotides that was later employed in the phylogeny reconstruction (illustrated on the uppermost of Figure  2 ).
DISCUSSION Several studies have proven the significant advantages of WGS approaches over traditional methods for the molecular disease characterization and further epidemiological surveillance of bacterial pathogens. Nevertheless, major technical difficulties are encountered not only with implementing WGS in clinical and reference laboratories, but when analysing the obtained data as well. The amount and complexity of data impose the requirement of specific bioinformatics skills. However, these tough challenges are largely overcome in the laboratories by employing automated and user-friendly bioinformatics pipelines. The numbers represent their prevalence within the analyzed dataset, that was established by sraX. In parentheses are shown the original values reported by  Tyson et al. (2018b).
These sequence files were selected from a set composed of 5,000 Klebsiella pneumoniae distinct genomes. We have chosen this organism since its genome size (≈5. 5 Mbp) is typical from a Gram-negative bacteria. Our results indicated a linear scale of running time (≈12. 5 min) and memory consumption (≈250 MB) with the processing of every 100-genome interval (Figures  3A, B , respectively). Defaults parameters were employed to ensure consistency during the testing. Each test was independently performed on a desktop computer Intel Xeon W-2104 processor (4-core, 3. 20 GHz) with 32 GB RAM. The fact of employing a simple desktop computer and completing the resistome analysis quite rapidly, envisages a greater performance when running sraX under more powerful computational resources. Afterwards, in order to assess the functionality and verify the accuracy of our proposed tool, we selected a genome dataset belonging to Enterococcus spp.
ACKNOWLEDGMENTS The author is indebted to the  Institut Français de Bioinformatique  and the  CONECT-AML (COllaborative NEtwork on research for Children and Teenagers with Acute Myeloid Leukemia) consortium  for their support. In addition, the author acknowledges  Mr. Mario R. Panunzi  and  MS. Brenda N. Henriquez  for their invaluable help and inspiration, and specially dedicates this paper to them.
Many empirical software engineering studies show that there is a great need for repositories where source code is acquired, filtered and classified. During the last few years, Ethereum block explorer services have emerged as a popular project to explore and search Ethereum blockchain data such as transactions, addresses, tokens, smart-contracts' source code, prices and other activities taking place on the Ethereum blockchain. Despite the availability of this kind of services, retrieving specific information useful to empirical software engineering studies, such as the study of smart-contracts' software metrics, might requires many sub-tasks, such as searching specific transactions in a block, parsing files in HTML format and filtering the smart-contracts to remove duplicated code or unused smart-contracts.
In this paper we afford this problem creating Smart Corpus', a Corpus of Smart Contracts in an organized, reasoned, and up to date repository where Solidity source code and other metadata about Ethereum smart contracts can easily and systematically be retrieved. We present the Smart Corpus' design and its initial implementation and we show how the data-set of smart contracts' source code in a variety of programming languages can be queried and processed, to get useful information on smart contracts and their software metrics. The Smart Corpus aims to create a smart-contracts' repository where smart contracts data (source code, ABI and byte-code) are freely and immediately available and also classified based on the main software metrics identified in the scientific literature. Smart-contracts source code has been validated by EtherScan and each contract comes with its own associated software metrics as computed by the freely available software PASO.
I. INTRODUCTION With the advent of blockchain technology as a mainstream technological innovation many researchers and software developers started investigating the new possibilities for software products relying on such infrastructure. Second generation blockchains offer the possibility to code the so called smart-contracts in a Turing complete programming language where all the main operations of traditional software systems can be carried on. The paradigmatic reference is the Ethereum blockchain which offers the possibility to deploy and execute decentralized applications (dApps) which are mainly coded in Solidity, presently the most adopted programming language  (1). Coding smart-contracts which run in a blockchain environment has its peculiarities and constraints and differs from coding in traditional out-of-chain contexts.
The novelty of the paradigm largely contributes to these faults, since developers do not have historical records or examples where to learn from previous code, as it happens in traditional software coding, where software reuse and coding by imitation are reference practices to help in coding better quality software. Another issue is the lack of reference measures, such as quality, complexity or coupling metrics, which are extensively used in out-of-chain software production to keep software projects under control  (4). The situation is slowly changing for what concerns historical records (even if history is quite recent) of software code, since the Ethereum blockchain can now count on up to 1. 5 million deployed smart-contracts, which have been used and running in the last few years. Access to the source code of this body of smart-contracts is still a challenge since the transparency and the open access granted by the public blockchains regards only data registered in the blocks, where only contracts' byte-code is available.
To access the smart-contracts' source code, the developers must resort to other means or to code repositories, such as the classical GitHub or similar resources. Fortunately, in the last few years, EtherScan 1 and other web sites have started providing smart-contracts checking as a service, so that Ethereum developers can submit their source code to be analyzed and the source code is made available afterword by the website. However, there is an odd side of the medal for many reasons: the access to this body of knowledge is far from easy and far from fast; it is not structured and organized; the smart-contracts metrics are not available and must be computed separately. All these tasks can and need to be automatized, to save developers' time and work, as well as computational resources. Indeed, in the last few years, a number of research papers have been published reporting findings based on smart-contracts' source code, mined from GitHub or some Ethereum block explorer such as EtherScan  (5; 6; 7; 8).
However, when conducting this kind of empirical research on smart-contracts with data from Ethereum blockchain, the tasks above mentioned need to be carried out by the developers themselves. The first task is the downloading of the smart-contracts' source code to be analyzed. One way to download smart-contracts' source code data is to inspect open-source software (OSS) project repositories such as GitHub  (9). Another way to perform this task is to use some Ethereum block explorer. These web services allow users to find the desired information by directly accessing the Ethereum blocks, by using a unique identifier or sequentially searching several blocks  (10). Some of these Ethereum block explorers provide RESTful Web services, which allow the users to obtain a JSON format payload containing various data.
These data may be related to a current or past state of the Ethereum blockchain: an example may be the list of transaction addresses included in a given block of the Ethereum blockchain. This activity might be tedious and time-consuming  (11) , when conducted by a single user/developer/researcher. Furthermore, the obtained smartcontracts' data-set can consist of duplicated smart contracts, i. e. smart-contracts having different addresses but with the same code. In this work we tackle these problems and propose an organized, easy to use, large and available software repository for Ethereum smart-contracts source code and metrics where users, researchers, blockchain start-ups and developers can take advantage of the body of knowledge collected during the last few years. This paper thus proposes Smart Corpus, a repository that provides the users with an interface, which allows to search and download smart contracts' source code.
The user interface is available at the following online address: https://aphd. github. io/smac-corpus/. The main challenge of the implementation lies in the fact that the Ethereum blockchain stores a massive amount of heterogeneous data, smart-contracts included, which enormously grow in time. For this reason, Smart Corpus was designed to be scalable, by adopting the latest cutting-edge technology, such as document-oriented database, graph query language and serverless computing platform  (12).
II. RELATED WORK A. Previous Literature on Software Corpus Analysis Gabel and Su  (13)  built and studied a corpus of open source software written in three of the most widely used languages: C, C++, and Java. The corpus contains six thousand software projects corresponding to 430 million lines of source code. The authors measured the degree to which each project of the corpus can be "assembled" solely from portions of the corpus, thus providing a precise measure of "uniqueness". Their primary contribution is to provide a quantitative answer to the following question: how unique is software? Our work also aims to answer this question, because many smart contracts written in the Solidity language have code that is a replication of other smart contracts, although presenting different addresses, as we will see in Section III-B.
Our goal is therefore to answer the question: how unique are smart contracts written in Solidity? in order to provide a corpus that is composed by smart contracts which can be distinguished from each other. Tempero and co-authors (14) presented the "Qualitas Corpus", a curated collection of open-source Java systems. The corpus reduced the time needed to find, collect, and organize the necessary source code sets to the time needed to download the corpus. The metadata provided with the corpus explicitly indicates the metrics calculated to identify the main features of the source code: the number of code lines, the number of classes, etc. Our work also aims to present a curated collection of smart-contracts equipped with a set of metadata with the aim of allowing experts in the blockchain field to perform static analysis.
By using this tool, he calculated and published software metrics results for 10,206 Solidity smart-contracts source code files written in Solidity languages. Our work also aims to calculate a set of metadata on the smart-contracts corpus by using a similar software. Pinna and colleagues (  16 ) performed a static analysis on 10,174 smart-contracts, deployed in the Ethereum blockchain. The authors showed that some metrics related to smart contracts, such as the number of transactions and the balances follow power-law distributions. Also, they reported that software code metrics in Solidity have (on average) lower values but higher variance than metrics values in other programming languages for standard softwares. Our work is inspired by their research as Smart Corpus is characterized by (some of) the metrics they defined, as we will explain in Section III-C.
C. Related Projects Other projects, similar to Smart Corpus, have been previously developed, to online access smart-contracts' code deployed in the Ethereum blockchain platform. The projects present specific features and limitations which are summarized in Table  I. • The smart contracts' source code collected in GitHub typically has not a direct reference to smart contracts deployed on the blockchain through an Ethereum address, therefore it is hard to find out whether it has been really tested or used on the blockchain. • GitHub does not implement a search engine to filter smart-contracts based on particular software metrics, such as the number of modifiers or payables. This is due to the fact that some metrics are specific to the type of language employed to write smart-contracts, i. e.
Solidity. • In GitHub there is no information on smart-contracts' use in a real blockchain scenario, on the number of transactions invoking smart-contracts, or the number of tokens associated with each smart contract. • GitHub does not provide smart contracts ABI or Opcode. It is highly probable that the users, especially if they are developers or researchers, want to access smart-contracts' source code choosing the features implemented in Smart Corpus, on the basis of its specific software metrics and its real usage on the blockchain. 2) Ethereum Block Explorers: Ethereum block explorers are platforms that allow the users to explore and search the Ethereum blockchain for transactions, addresses, tokens and other activities taking place on the Ethereum blockchain  (20). Unlike GitHub, the Ethereum block explorers allow accessing only Ethereum data used in the Ethereum blockchain and thus smart contracts' real use-cases.
III. RESEARCH METHODOLOGY Smart Corpus has been designed to provide the users with a reasoned repository, i. e. a repository which is not just a web space where to collect them but also and mainly a service to help the researchers to filter and analyze the Smart contracts' source code. To this aim, Smart Corpus has been planned to perform four main automatic operations on Smart contracts' source code (data): 1) Data Retrieving, 2) Data Cleaning, 3) Data Modelling, 4) Data Querying. Figure  1  shows the Smart Corpus's pipeline of the operations. We collected smart-contracts' source code, smart-contracts' application binary interface (ABI) and smart-contracts' byte-code through the EtherScan web site, which makes available the source code of a subset of verified smart-contracts deployed on the Etehreum blockchain, though in a laborious way.
We instead made this task easier and automatic via a retrieving data script available at the following address 10. During this phase the blockchain blocks are automatically inspected. Each block is formed by a list of transactions between two different blockchain addresses, which can refer to a wallet or to a smart-contract. The script looks for addresses that refer to a smart-contract and when the source code is available, it downloads the smart-contract's source code, the ABI and the byte-code. The data coming from the source code are not immediately available as they are embedded in the HTML code of the web page provided by EtherScan. Therefore, the script removes the HTML tags and stores the code cleaned up. Figure  2a  shows how Smart Corpus finds the smart-contracts' list in a given block.
C. Modelling Data Unlike the tools discussed in the related work Section II, Smart Corpus associates different metrics (intrinsic metrics and extrinsic metrics) to the smart-contracts, aiming to facilitate the selection of a smart-contracts' set that meets precise requirements. The metrics associated with the smart-contracts are then stored in a document-oriented database. Figure  3  shows the database schema of a smart contract. Fig. 3 : Smart-Corpus's database schema 1) Smart Contracts' Intrinsic Metrics: The smart contracts' intrinsic metrics are smart-contracts' software metrics which depend on internal properties of the smart contracts' code, such as the number of lines of code, modifiers, payable, etc. Table  II  shows the smart-contracts' intrinsic software metrics. 2) Smart Contract Extrinsic Metric: The smart-contracts' extrinsic metrics are properties depending on external factors rather than the code itself, such as the number of transactions executed to the smart-contracts or the number of tokens associated with the smart contracts.
D. Filtered Data The smart contracts' source code is stored in a file system and is organized in folders and subfolders to ease the navigation. Figure  4  shows the sub-directory structure. The first leaf corresponds to the first two letters of the smart contract address and then each directory contains the file having as name the full address of the smart contract and three different extensions, respectively. sol for the Solidity source code,. abi for the ABI, and. bytecode for the byte code. The metadata (both the intrinsic and extrinsic metrics) are stored in a document-oriented database: MongoDB  (22). The choice to use a Document-oriented database instead of a relational database such as Mysql is based on: • Relational databases are prone to deterioration when data-sets overcome a size threshold, while a document-oriented database such as MongoDB comes with inbuilt load balancer, which makes it a better solution in applications with high data load  (23).
We update MongoDB each day to generate the data archive. • Unlike relational databases where data is stored in rows and columns, document oriented databases store data in documents. The documents typically use a structure similar to JSON (JavaScript Object Notation), they indeed provide a natural way to model data that is closely aligned with object oriented programming. Each document is considered as an object in object oriented programming, similarly each document is a JSON in document-oriented database. The concept of schema in document databases is dynamic: every document might contain a different number of fields. This is useful when modeling unstructured and polymorphic data. Also, document databases allow robust queries: any combination of fields in the document can be combined for querying data  (24).
• Below the form, the smart contracts filtered by the user are displayed. For readability reasons, only a part of the smartcontracts metrics are presented in the table layout format. Each column header in the table indicates the name of a metric associated to smart-contracts. While the HTML-GUI displays just some metrics, the user can access to all the metrics and to the smart-contracts' source code by selecting the checkbox displayed on the right of the smart-contract address and clicking on the red button "download". The user can also access the original repository where the smart-contract was retrieved, i. e. the EtherScan service. 2) Smart Corpus GraphQL application: Graph Query Language (GQL) is a full data query language to implement webbased services, centered on high-level abstractions, such as schemas, types, queries, and mutations.
F. Use Case A use case for Smart Corpus might concern a researcher interested in the static analysis of smart contracts. For example, the researcher might be interested in performing an analysis of smart-contracts written with a particular version of the Solidity language, 6. 0, and having at least a payable function in the smart-contract. The research of smart contracts that meets these requirements would be very expensive in terms of time, work and computational resources, using a service like EtherScan. Instead, thanks to the Smart Corpus, the user needs to perform only a few steps, as described below: • connect to the service through the link: https://aphd. github. io/smac-corpus/ • select the option "version 6. 0" from the drop-down menu entitled "pragma version". • select the option "greater than zero" from the drop-down menu entitled "number of payables".