dataesr/scientific-paragraphs-categorization huggingface dataset:
New York City subway metagenome (NYCS) Afshinenkoo et al. [1]  explored the microbial diversity of New York City subways by analyzing read-sets from multiple metagenomic samples (that we refer to as the NYCS dataset). Although this study did not specifically pursue the goal of finding IPGs, Parks et al. [53]  identified a metagenome-assembled genome B. thuringiensis UBA3967 in some of these samples (Illumina read-set with accession number SRR1748627). Gillis et al. [26]  analyzed the B. thuringiensis UBA3967 strain and noticed that, although it is very similar to the Bti genomes, it cannot be classified as a Bti strain since it lacks the Bti plasmids. Since all known Cry-encoding genes reside on plasmids, Gillis et al. [26]  came to a conclusion that the reported B. thuringiensis UBA3967 assembly does not encode the entire set of Bti Cry toxins.
Analyzing IPGs predicted by ORFograph We used Bti1, Bti2, B ALL , and NYCS datasets to analyze ORFograph predictions. For each dataset, we launched SPAdes (Bti1, Bti2, and B ALL datasets) or metaSPAdes (NYCS dataset) to construct the assembly graph and further launched the ORFograph pipeline applying SPAligner (using all known Cry, Cyt, and Vip toxins) and PathRacer (using all known HMMs derived from these toxins). Supplementary Table  S1  provides information about ORFograph runtime and memory footprint. Shikov et al. [61]  recently developed the CryProcessor pipeline for IPG discovery by applying the PathRacer tool  [62]  to an assembly graph. However, since CryProcessor only searches for the three-domain Cry toxins, it cannot be used as a general pipeline for IPGs discovery from assembly graphs.
ORFograph results for Bti1 and Bti2 genomes Previous studies identified two genes encoding Cry proteins in Bti1 and seven genes encoding Cry proteins in Bti2  [8]. Both Bti1 and Bti2 have the pBtic100 plasmid carrying cry60Aa and cry60Ba genes. In addition, Bti2 has the pBtoxis plasmid with five Cry genes (cry4Aa, two cry4Ba, cry10Aa, cry11Aa) and three Cyt genes (cyt1Aa, cyt2Ba, cyt1Ca). ORFograph identified 17  (23)  clusters of putative IPGs in Bti1 (Bti2). We selected cluster representatives and ran BLAST  [2]  against the non-redundant protein database. Although most of these representatives have > 99% identity with thioredoxins, metallophosphoesterases, and disulfide reductases, ORFograph also identified cry60Aa and cry60Ba toxins in Bti1 that each resided in a single contig (and thus can be found without using ORFograph).
3. This filtering is based on the concept of "unique edges" that are currently defined based on a fixed threshold of 300 nt (see the "Methods" section), pointing to the challenge of parameter selection in ORFograph. Since the Bti2 dataset resulted in a highquality assembly (N50 = 157kb), increasing this threshold for assemblies with high N50 (e. g. , from 300 nt to 500 nt) would lead to identifying cry4Aa. In the future, we will modify ORFosearch to make this threshold variable rather than fixed. As most Cry genes reside on plasmids, we also checked if the specialized plasmid assembler plasmid-SPAdes  [3]  can resolve the cry4 gene-paths in the Bti2 assembly graph. We launched plasmidSPAdes with default parameters on Bti2 reads and restored the same set of Cry genes as with the standard assembly approach.
ORFograph results for B ALL datasets ORFograph identified 72 datasets (among all 342 B ALL datasets) that have IPGs alignments scattered over multiple contigs and selected 419 novel cluster representatives among them (Supplementary Table  S2 ). One of B ALL datasets (SRR6238356) contained a very large number of potential IPGs, most of which arise from traversing a highly complex area of the assembly graph (Fig. 1 ). While this dataset greatly increased the number of potential IPGs (increasing it from 2488 to over 5600), it had only 40 representatives, that were included in the follow-up analysis. Unclassified Cry1, Cry1A, and Cry1C are the most productive Cry families that served as seeds for identifying most IPGs (≅ 3522 ORFs and ≅ 122 cluster representatives in total). CW_binding_1 and Bacillus HBL are the most productive HMMs (Fig.
We classify a reliable IPGs as novel if it has AAI exceeding 90% (but less 100%) to a known IPG from the BLAST database. Three hundred nine out of 419 cluster representatives were classified as reliable and 232 of them were classified as novel (Fig. 5  and Supplementary Table  S3 ). One hundred forty-five out of 232 novel IPGs have the length of 1000-1200 amino acids (Fig. 6 ). Sixty-seven candidate with length around 750-900 bp are similar to Vip1B, while other 20 candidate IPGs shorter than 1000 bp are similar to Cry2, Cry4, Cry5, Cry11, Cry13, Cry27, and Vip3 families. Most of 145 longer sequences have high identity with sequences from Cry1 family (Fig. 6 , left) and only 8 of them are similar to Cry4, Cry5, and Cry9 genes. Figure  6 , right illustrates that just three datasets (SRR6238356, SRR6238345, and SRR4996216) contain 77 out of 232 reliable IPGs.
The first two of these datasets were generated in Méric et al. [43]  where the authors analyzed the transfer of Cry-rich plasmids between various species. Méric et al. [43]  analyzed assemblies of 190 B. cereus group isolates, identified genes encoding Bt toxins (Cry, Cyt, Vip, and Sip) using BtToxin_scanner, and revealed that most Cry gene variants belong to Cry1Ia2, Cry2Aa9, Cry2Ab3, and Vip3A families. Our analysis of these datasets is consistent with the results in Méric et al. [43]  with respect to IPGs found in single contigs. However, neither of 59 potential IPGs identified by ORFograph was identified by BtToxin_scanner (Table  S4  in  [43] ). In addition, two potential IPGs identified by BtToxin_scanner represent fragments (rather than complete genes) of the reliable IPGs identified by ORFograph.
SRR4996216 dataset (with 17 identified IPGs) contains Illumina reads from B. thuringiensis serovar aizawai strain HD-133 assembled via the A5-miseq pipeline  [14]  in the original study  [34]. Genome annotation was carried out using Prokka  [60]  and the NCBI's Prokaryotic Genome Annotation Pipeline. The original study revealed Cry1Aa, Cry1Ab, Cry1Ca, Cry1Da, Cry1Ia, Cry2Ab, and Cry9Ea, which include many partial or apparently fragmented genes. In contrast, ORFograph found 16 full IPGs, including novel variants of Cry1Aa, Cry1b, Cry1Ca, and Cry1Da. All Vip1B-like IPGs were obtained from datasets generated in  [41] ), where authors characterized the pathogen genes involved in coevolutionary adaptation in an animal host-pathogen interaction system of Caenorhabditis elegans and Bacillus thuringiensis.
Our analysis of the diverse B ALL datasets demonstrates that ORFograph greatly extends the set of IPGs found in previous studies. Although our analysis is not exactly benchmarking (since we are identifying sequences not previously assembled), Supplementary Figure  S2  present a comparison of putative novel IPGs to existing annotations in the IPG database (the labeled nodes on the tree show known IPGs and the others show putative novel IPGs), illustrating the value of ORFograph as a tool for novel IPG discovery. cluster representative with the non-redundant protein database using BLAST. Similar to the analysis of genomic datasets, most cluster representatives have > 99% identity with thioredoxins, metallophosphoesterases, and disulfide reductases. However, ORFograph also identified cry60Aa, cry60Ba, cry11Aa, cry10Aa, cry4Aa, and two cry4Ba genes as well as three Cyt toxins.
Similar to the Bti2 dataset, since cry4 genes were scattered over several contigs (Fig. 7 ), they would evade identification without ORFograph. Since the NYCS dataset contains many Cry and Cyt genes that are similar to genes identified in the Bti2 dataset, we can conclude that UBA3967, also assembled from this dataset, represents mostly the chromosomal part of a viable Bti strain, but with only traces of plasmids that should also be there. We thus detected the corresponding toxins encoded by the plasmids, as it should be expected.
Methods Constructing database of known IPGs and their Hidden Markov Models (HMMs) Accession numbers of known Cry and Vip toxins were taken from the Bt nomenclature list (Crickmore et al. 2018), and sequences were downloaded from the NCBI database, forming the IPG database. This procedure resulted in extracting 941 protein sequences (811 Cry and Cyt toxins as well as 130 Vip toxins) with the average length of each protein approximately 800 amino acids (Fig. 8 , left). In addition to forming the IPG database, we selected 66 publicly available HMMs commonly found in IPGs and downloaded them from the Pfam database  [45]. The number of hidden states in the selected HMMs (referred to as HMM length) varies from 25 to 350 states (Fig. 8 , middle).
Analyzing HMM positions within IPGs We aligned all HMMs to the IPG database using the HMMer tool  [22]  and constructed the distribution of their starting positions inside the IPGs. ORFograph uses this distribution to find the most likely distance from the start codon to the position of the HMMs within an IPG. We aligned each HMM to all known IPGs and generated a histogram of all distances from the gene start to the start of each full HMM alignment. As Supplementary Figure  S3  illustrates, this distance is very conserved for some HMMs (e. g. , PA14) and less conserved for other HMMs (e. g. , Aegerolysin). We thus defined the likelihood of a given distance d as the fraction of distances in the interval [d-binSize, d+binSize] in the histogram (the default value binSize=150). Using the computed likelihoods, ORFograph derives a set of the most likely distances from the start codon for each HMM alignment in the assembly graph and uses these distances to find the most likely prefixes as the prefixes with likelihoods exceeding the default threshold (a similar procedure is used to find the most likely suffixes).
concluded that the Number of novel IPGs per dataset 40 30 Vip1Bb Vip1Ba 20 10 0 SRR6238356 SRR6238345 SRR4996216 SRR634070 SRR633907 SRR6238358 SRR642765 ERR498200 SRR642768 SRR633679 SRR6238354 SRR6238355 SRR634049 SRR642767 SRR2175480 SRR642766 ERR498198 SRR7165863 SRR6238348 SRR6238351 SRR6238360 ERR1103992 SRR6238346 SRR9189225 SRR9189593 SRR8846359 SRR8467560 SRR2147684 DRR160017 ERR498179 ERR498186 ERR498180 ERR498164 ERR498165 ERR498166 ERR498167 ERR498187 ERR498188 ERR498199 SRR6238350 Cry Vip.
In the logistic chain domain, the traceability of shipments in their entire delivery process from the shipper to the consignee involves many stakeholders. From the traceability data, contractual decisions may be taken such as incident detection, validation of the delivery or billing. The stakeholders require transparency in the whole process. The combination of the Internet of Things (IoT) and the blockchain paradigms helps in the development of automated and trusted systems. In this context, ensuring the quality of the IoT data is an absolute requirement for the adoption of those technologies. In this article, we propose an approach to assess the data quality (DQ) of IoT data sources using a logistic traceability smart contract developed on top of a blockchain. We select the quality dimensions relevant to our context, namely accuracy, completeness, consistency and currentness, with a proposition of their corresponding measurement methods.
The lack of transparency in the qualification process could also be a source of dispute on the correct application of data handling and qualification rules agreed by all the traceability process stakeholders. The advent of blockchain technology and smart contracts help develop new traceability systems. Such systems allow stakeholders to achieve the secure and transparent sharing of traceability data, using the blockchain secured and distributed ledger. In addition, smart contracts allow stakeholders to share data handling and decision-making rules, in order to ensure that the same agreed rules are applied by all the stakeholders. Increasingly, IoT devices are used to automatically collect field data. Those data are used both for traceability purpose and to take automatic decisions, such as the creation of shipment incidents, when one or more of the negotiated shipment transport conditions are not respected.
As a result, the human intervention is limited in the process, as well as process error probability. To automate the traceability process decision making, new traceability system architectures have been proposed in the literature combining smart contracts and IoT (see, e. g. ,  [1] [2] [3] [4] ). However, in the existing smart contracts and IoT traceability systems literature, many of the provided architectures propose to integrate the IoT data directly into the smart contract (see, e. g. ,  [1] [2] [3] [4] ). This could lead to unsound decisions taken by the smart contract based on erroneous data collected and sent directly to the smart contract by the IoT data sources. To overcome this issue, we propose to introduce an IoT data qualification process in smart-contracts and IoT-based traceability architectures.
We proposed in  [5]  to enhance traceability architectures using blockchain, smart contracts and IoT, combined with a lightweight IoT data qualification process. However, in this previous work, the proposed qualification process covered only outlier measure detection, which is only one facet of data quality. Furthermore, we did not compute the qualification at different levels such as the measure and the sensor level. Moreover, the IoT data qualification process was centralized at one stakeholder's site, and there was no guarantee for the other stakeholders on the correct execution of the agreed IoT data qualification rules. In addition, the stakeholder in charge of the IoT data qualification represented a single point of failure of the architecture on the IoT data qualification part. To overcome the above limitations, the main contributions of this article are threefold: (i) The literature review of IoT data qualification highlights that the data quality of a system is assessed by means of several dimensions.
Related Works Data quality is not a recent research topic. The first data quality studies concerned databases. Many data quality aspects have been considered such as the accuracy, consistency and reliability to improve the quality of data inputs into databases and handle databases incompatibility and time critical delivery data  [6]. With the advent of the IoT as new data sources, the existing data quality studied aspects needed to be extended to the specificities of those new data sources. The data collected from IoT data sources need to be controlled even more due to the limited capacity of these sources to ensure the security and the quality of their data. The "Never trust user input" should evolve to "Never trust things input", as stated by Karkouch et al. [7]. Moreover, the emergence of blockchain opens new opportunities for systems that involve multiple stakeholders.
This connected object id is used by the smart contract to link the received IoT messages to the right shipment in the smart contract. In this context, the data are pushed by the IoT object. The pull/push of data from/to the connected object is out of the scope of our work. The shipment connected object collects data about the shipment pickup, transport and delivery conditions. Each stakeholder could declare other IoT data sources, such as IoT data sources related to factories, warehouses, transport vehicles, etc. In general, every data source that can collect and send automatically measurements about the shipments could be declared by the stakeholder as an IoT data source. Moreover, all IoT data sources, except the shipment connected object, help to collect data about the shipment conditions in a specific segment of the transport operation.
Evaluation The objectives of this section are: (i) to evaluate the proposed quality measures; (ii) to evaluate the impact of the IoT data quality module on the number of created incidents; and (iii) to evaluate the impact of the IoT data quality module on the IoT data event insertion time in the blockchain. We evaluated our proposed quality measures to measure their pertinence and performance. We also monitored the number of quality incidents created to highlight the impact of the quality module. The number of shipment incidents was also monitored to emphasize the impact of the quality module on the business decisions. The IoT data event insertion time in the blockchain was also measured in our tests, firstly with the quality module activated and then with the quality module inactivated, in order to evaluate the impact of our proposed quality module on the data event insertion time and ensure the final users that this time is acceptable while ensuring the quality.
Smart Contract Architecture For the implementation purpose, we used the same architecture used in our previous work on the traceability using smart contracts and IoT  [5]. It is an architecture based on the use of Hyperledger Fabric as the blockchain implementation, with three peers (stakeholders): a shipper, a carrier and a consignee. On the top of this blockchain, we implemented our traceability and IoT data qualification smart contract. The smart contract used in this evaluation was developed on the top of a Hyperledger Fabric blockchain, using the Fabric Java Framework. We used in this evaluation a Virtual Machine (VM) with the characteristics depicted in Table  3. Furthermore, we set the Hyperledger Fabric block creation timeout to 1 s and the maximum number of transactions per block to 15.
This means that, after the reception of a new transaction, the system will trigger the block creation either after a time wait of 1 s or after a total number of 15 new transactions is reached. In addition, we used in this evaluation the Raft consensus algorithm, with a unique ordering service node  [5]. In the existing traceability smart contract [5], we added many new methods such as createDataSource and assignDataSource. The createDataSource method inserts the data source given as input in the blockchain. The assignDataSouce method assigns an existing IoT data source to an existing shipment, using their IDs. Based on the quality measures proposed in this article, we updated the addIoTEvent method with the following new functionalities: (i) calculate the event quality measures; and (ii) update the IoT data source and the quality measures of the shipments related to this IoT data source.
Evaluation Experimental Choices Due to a lack of real data to evaluate the proposed architecture in our use case, we chose to simulate our use case data with a well-known dataset in the IoT domain. The Intel Berkeley dataset is a collection of sensor data, collected by Intel research team in the Intel Berkeley Research lab, between 28 February and 5 April 2004  [35]. An example of the dataset content is depicted in Table  4. To adapt this dataset to our context, we considered every sensor as an IoT data source. This gives us 54 data sources to be handled. For the shipments, we used every 24 h of sensor data collection as a shipment, which results in 2052 shipments (54 sensors multiplied by 38, the number of data collection days), for the whole dataset. Furthermore, we considered only the temperature measures in this evaluation because it is the main measure for our use case, but the module could be used to handle any other measure type.
For the shipment incidents, we chose an accepted temperature interval of 20 to 25 • C based on the work of Hui et al. [36]. Beyond this temperature interval, if the received event quality is compliant with the shipment quality threshold, this event results in a shipment incident created for all the shipments that have an active assignment relationship with the event data source. There was no information in the dataset about the sensor's precision value. Consequently, we chose to set this value to 0. 5 • C, which is a recurrent value in the temperature sensors. In the following evaluation results, we did not take into account the sensor 5 from which we did not see any event. We also ignored some other events with the sensor IDs 55, 56 and 58, because in the dataset reference the number of sensors was only 54, and events coming from the same sensor with the same event number (113,474 events in the dataset).
There were also 355 events in the dataset that we could not parse correctly due to their data presentation errors and 526 incomplete lines, from which we could not get all the event required data. This results in a total of 2,199,327 events integrated correctly in our quality tests, from a total of 2,313,682 events present in the dataset. We used the event timestamp in the dataset as an event reception timestamp in this evaluation. Moreover, we used this timestamp to order and identify the events, for shipment incident creation and closing purpose. The results of this choice were 10,299 duplicated events, because they had the same timestamp as previously received events from the same sensor. Furthermore, we use the quality threshold to define the stakeholder's requirement for the quality indexes of events to be integrated in the data source or sent to shipments.
Results Concerning the Accuracy, Completeness and Currentness Dimensions Firstly, regarding the accuracy, the sensors used to collect the Intel Berkeley dataset, a valid temperature value should be in the range of 0-50 • C according to  [37] , otherwise we consider this temperature as inaccurate. Regarding the completeness, we used the following parameters: the update interval of 31 s, the maximum timestamp among the already integrated events timestamps, the start IoT data source and the shipment start timestamp. We set the IoT data source start timestamp at 28 February 2004 at 00:00:00 am, and, for the shipment, the start timestamp is the shipment date and the start time set at 00:00:00 am and the end at 11:59:59 pm. Concerning the currentness, we used the measure interval of 31 s given for the dataset.
g. , 0. 9), there is no usable sensor in this dataset. Thanks to the quality module, all the events with a quality incident problem are not integrated into the shipments assigned to the event data source, and this keeps the shipment events quality at the level fixed and agreed by all the stakeholders. For example, in the case of Sensor 45, when we set the quality threshold at 1, 9% of the events received from this sensor have not been integrated into the source related shipments, due to their quality problems. In Table  7 , we can clearly see the impact of the threshold choice on the percentage of quality incidents. This percentage represents the events that do not respect the agreed quality thresholds. The events are filtered at the data source level according to the selected quality threshold value.
Impact of the IoT Data Quality Module on the IoT Data Event Insertion For the smart contract IoT data quality evaluation, and due to our blockchain architecture response time (around 1 s per operation), we selected a sample of 3000 events from the dataset. This sample corresponds to the first 1000 events received from the Sensors 1-3 on 28 February 2004. The average response time of the addIoTEvent using the 3000 events data sample was around 1. 7 s, with an average standard deviation of 0. 174 s. When we disabled the quality module, with the same data sample, the average response time of this method drops to around 1. 6 s, with an average standard deviation of 0. 158 s. This result shows that our quality module adds only around 0. 1 s to the event integration time. The additional quality module cost is acceptable regarding the data quality improvement brought by this module.