dataesr/scientific-paragraphs-categorization huggingface dataset:
Then the block is committed into the ledger  [63]. Every peer has a copy of the ledger and a CouchDB state database. Furthermore, every peer includes smart contracts used to control access to the off-chain database and to write transactions to the blockchain ledger. Likewise, CouchDB presents actual data stored in a particular moment of time with rich query functionalities and also supports modeling of smart contracts as JavaScript Object Notation (JSON). In addition, the hyperledger framework presents a deployed blockchain business network that is manipulated by a set of tools called hyperledger composer  [33]. The latter includes playground and composerrest-server. In fact, the playground provides a graphic interface for the configuration, deployment, and testing of a blockchain business network.
Likewise, composerrest-server is used to create from a Business Network Archive (BNA) file a REST API that is consumed by HTTP or REST Client. Thus, hyperledger composer-rest-server exposes blockchain services to the client application through REST API. In addition, Google fabric client using Remote Procedure Calls (gRPC) communicates with the fabric network  [64]. Besides, the blockchain network generates notifications to the client via WebSocket  [65]. In our system, smart contracts were designed and implemented via utilizing hyperledger composer. A smart contract is a self-executing computer program intended to perform a designed feature. It is defined within a chaincode and is compiled in the form of a BNA. Furthermore, a smart contract consists of four main parts including the model, the logic of transaction, the access control rules, and query definition.
In fact, the model includes participants, assets, and transaction definition. The participants are the users that interact with the blockchain services. The assets are tangible or intangible things used between the participants and saved in blockchain registries. Transactions are logical operations defined in the smart contract. The fabric uses access control lists (ACLs) for resource access management that provides authentication and authorization to the participants and defines the roles of each participant within the business network. The fourth part of a smart contract is queries written in the bespoke query language in a separate file. The main responsibility of the queries is to select customized data based on user requests. We propose to use OrbitDB with the Inter-Planetary File System (IPFS) as an Off-chain database.
In fact, an off-chain database is used in order to store UAV's collected data that is too large to be stored in the blockchain efficiently. OrbitDB is a distributed peer-to-peer database performing with IPFS. The latter generates the hash of the committed data into OrbitDB and stores it as an immutable transaction in the blockchain. When requesting to retrieve data from the off-chain database, access control policies will be executed by smart contracts (chaincode) to allow or deny data sharing. In IPFS, the uploaded files are fragmented. The maximum size of a fragment is 256 kilobytes. The fragment might contain only data, data, and links to other fragments, or only links. Every fragment is identified by a cryptographic hash also named content identifier. In order to reconstruct any file from its fragments, a Merkle directed acyclic graph (Merkle DAG) is used  [66].
The key idea is that Hyperledger uses identity management and access control lists via private channels, allowing nodes to control and limit the access to their distributed sensitive information within the blockchain network. Through this approach, peers in the network know each other by their public identities, but they do not need to know the information disseminated over the network. In addition, we investigate the use of an off-chain database to store sensitive UAV data in order to preserve the privacy of UAV data and to mitigate the storage burden on UAVs. This method is suitable to manage massive sensitive UAV data as they should have access restrictions because saving them inside the blockchain is not an efficient solution. Peers in the IoD can validate UAV data without the need of a central authority for approval, only by checking the blockchain, and UAV data is stored securely in the off-chain database.
Conclusions In this work, we have proposed UTM-Chain, a blockchain-based security solution for an unmanned traffic management system. Blockchain technology has been integrated with UTM components to build an efficient and secure traffic monitoring system. This work also provides a road map towards efficient and secure management of UAV flight data records. The proposed architecture is based on a permissioned blockchain network. In the proposed architecture, blockchain is not used only for storing transactions but also it is used to control access to the flight data stored in a decentralized off-chain database through the execution of smart contracts. A decentralized off-chain database (OrbitDB with IPFS) and decentralized blockchain network improve the security level of our proposed architecture as centralized security issues are mitigated.
• Users: are third-parties that have access to the parameters and data of the UAVs and GCS for useful purposes. •Blockchain Network: The main role of the blockchain network is to maintain a distributed immutable database of actions performed by users, data collected from drones, and commands from GCS. The recorded transactions are shared between nodes in the network. In our architecture, the users, UAVs, and GCSs act as nodes, storing the whole blockchain and participating in the consensus protocol to verify blocks. • Cloud server: is responsible for offloading computation from the drone to overcome the computing resources' limitations of the UAVs and optimize the mission execution. Correspondingly, the flying time of the drones can be extended.
[42] (2017) Securely relay drone information via blockchain Public and private keys, UAVs, BC UAVNET Smart contracts Y N Y Cannot be applied to IoD It is not practical Liang et al. [43] (2017) Ensure the secure communication and integrity of data collected from drones using blockchain and cloud storage Chainpoint, Ubuntu, Apache The network is composed of drone, control system, blockchain network, cloud database, and cloud server POW, bitcoin Y Y Y to minimize the drones' burden by using the traditional cloud server together with blockchain technology Theoretical Explanation Han et al. [38] (2017) Encrypt drone-GCS communication Elliptic Curve Crypto-graphy (ECC) and RSA The network is composed of one UAV and a ground station N/A N N N Public key cryptography depends on the third-party authority.
Testbed setup In the following, we depict our Proof of Concept of the proposed architecture. As illustrated in Fig. 7 , our deployment testbed is built over two physical hosts that represent two different domains. Note that in the following, the terms domain and server can be used interchangeably. We also leverage on LXC  [33]  containers that are used in order to host the different Service Functions connected through OVS (Open vSwitch)  [34]  switches that ensure packet forwarding using OpenFlow rules. Version 2. 11. 0 is used in order to support the NSH encapsulation. The physical hosts are interconnected using VXLAN tunnels. Packet processing is performed by the classifiers, IBNs and VNFs using the Python-based library scapy  [35] , and the traffic is generated using the hping3 packet assembler  [36].
We implemented, using Python scripts, our multi-domain orchestrator that performs NSD partitioning, as well as local domain orchestrators that ensure the sub-SFC deployment. In addition, we used a basic SDN controller that will construct the classification and forwarding rules for each SFC, and enforce them at the switch level, as well as the classifiers and IBNs at the domain's borders. The hardware and software setup is outlined in Table  1. In order to support Service Chaining, two different encapsulation methods are used, a header-based encapsulation with the NSH header, and a tag-based encapsulation with MPLS Segment Routing: • The Network Service Header  [25]  encapsulation is illustrated in Fig. 8 , the SFC Identifier is encoded into the 3-byte Service Path Identifier (SPI) field, and the position of the packet inside the chain is encoded into the Service Index (SI) field; this tuple allows the SFFs to determine the next hop for the packets.
This header is added upon the classification of packets, and each time the packet passes through a Service Function along the chain, the SI is decremented. In this implementation, we use the NSH MD-Type 1, which is the fixed-length metadata type, therefore, the Network Service Header's size is fixed to 24 bytes. • The Segment Routing  [37]  encapsulation is depicted in Fig. 9 , where the classifier encapsulates the forwarding path into a packet as a list of hops, therefore reducing the amount of state information that needs to be stored on the forwarding devices. In our implementation, we leverage on Multi Protocol Label Switching (MPLS) headers, where each Service Function ID is encoded into a 20-bit MPLS label. The SFFs determine the next hop by reading the label field of the outer MPLS header, and each time a packet passes by a Service Function, the outer MPLS header is stripped off, so that the packet can be routed to the next SF in the SFC.
Encapsulation mapping If we consider the scenario where the WAN domain is SFC-aware, the IBN strips off the higher level encapsulation of packets at each domain's entry and replaces it by the domain's encapsulation for the identified sub-chain; the IBN should also be able to retrieve and restore that same higher level encapsulation at the egress of the domain. Note that the higher-level encapsulation might contain specific metadata that could not be restored by a simple re-classification of the packets by the IBN. The hierarchical SFC document by the IETF  [30]  lists a few methods that could be envisaged to retain these metadata such as header nesting, flow state saving, or pushing the upper level encapsulation into the metadata field of the lower level encapsulation. However, at the time of this writing, these methods have not yet been evaluated, and new ones could also be proposed by future research works.
ITSx identifies the highly conserved neighbouring rRNA sequences SSU, 5. 8S and/or LSU. If the targeted variable region (ITS1 or ITS2) is not detected, the sequence is discarded. The ITSx step is time consuming and has to be done after clustering and filtering steps. FROGS Affiliation: users have to select one of available ITS databanks. The affiliation process is specific for ITS, adapted to un-merged data thanks to a mix of local and global alignments. Because of the inclusion of the 100 N stretch in the "combined" sequences, the affiliation of the OTU seed sequences against the databases by BLASTN+  [17] , as in FROGS v1, was not possible. BLASTN+ aligns the sequences locally and, by definition, only keeps the best local alignment, i. e. , either the 5' sequence or the 3' sequence, but in no case the entire "combined" sequence, since the N stretch collapses the BLAST score.
We thus chose a double strategy. All normally merged sequences are treated like in FROGS v1 by BLASTN+ against a databank (e. g. UNITE) while the combined sequences, which are automatically detected by the system, are treated with needleall  [18] , a global sequence aligner. To align with needleall takes too long, to reduce this time, we chose to align combined sequences not against the entire databank (e. g. UNITE) but against a drastically reduced bank. To this end, first, the two sequence parts and for all: the absence of the full or partial taxon name. Two modes are possible to process these impacted OTUs. In the masking mode (--mask), the taxonomic information is replaced by NA. In this case, affiliations with insufficient characteristics are not retained. In delete mode (--delete), the OTUs are deleted according to the filter settings.
DATASETS AND TESTS TO ASSESS THE MANAGEMENT OF ITS: To test our methodology, we generated increasingly complex simulated datasets, available on https://doi. org/10. 15454/AOT7UL (supplemental data section 3, supplemental Figure  1 ) and used a mock biological community (supplemental data section 4). These data were processed by FROGS but also by USEARCH, QIIME2 and DADA2. Since QIIME2 and DADA2 filter out nonoverlapping reads, we used QIIME2 and DADA2 in single-end and paired-end modes. For USEARCH, we followed the original guidelines by taking into account merged sequences and 5' R1 reads of non-overlapping paired-end sequences. All benchmarking command lines are available on http://frogs. toulouse. inrae. fr/ITS/frogs-its. html#Command_lines. We used four metrics to assess our results: (i) the divergence rate: computed as the Bray-Curtis distance between expected and observed abundance profiles at a given taxonomic level (observed composition is the abundance resulting from each tool process), (ii) the number of falsenegative taxa (FN): the number of expected taxa that were not recovered by the method, (iii) the number of false positive taxa (FP): the number of recovered taxa that were not expected, and (iv) the number of true positive taxa (TP): the number of recovered taxa that were expected.
( On real data (http://frogs. toulouse. inrae. fr/ITS/frogs-its-meat. html, supplemental Figure  7  and supplemental Table ), FROGS produced very good results. The divergence rates of all the tool were worse than on simulated data because they were calculated on expected data while it is difficult to match theoretical and real quantities when preparing samples. The greatest divergences could therefore be due to this. Concerning FP, TP and FN, FROGS showed high precision and a satisfactory recall rate (median 0. 99 and 0. 92 respectively), even if, as expected, FROGS was not able to differentiate between species that are too similar, e. g. Penicillium. Nevertheless, considering the taxonomies found, FROGS was the tool that detected the most expected species. FROGS also reconstructed the expected sequences best (100% identity and 100% coverage of the entire sequence).
CONCLUSION FROGS is an easy-to-use suite that is perfectly suited and efficient not only for the processing of internal transcribed spacer amplicon sequences, but also of all other unmergeable amplicons i. e. rpb2  [19]  and D1-D2  [20]. The test of FROGS data processing demonstrated excellent recall rates and precision thanks to smart ITS data management while accurately reconstructing the sequences as they are expected. FROGS can be used both on the command line and on the Galaxy interface, making it easy for everyone to access.
NINE FROGS STATISTICAL TOOLS The analysis of OTU (Operational Taxonomic Unit) tables with respect to the associated metadata (pH, temperature, weight, treatment, etc. ) requires the use of statistical methods dedicated to compositional data. The OTU abundance table produced by FROGS contains the abundance information for each OTU in each sample as well as its taxonomic affiliation. Questions such as "How many species are there in each sample i. e. , species richness?" and "Which OTUs are abundant in my samples, i. e. structure" or "How does the microbial community vary with the different metadata?" can only be based on the calculation of distances, indices and descriptive statistics. FROGS therefore has seven tools for the statistical analysis of samples using the phyloseq R package, extended R functions in Rmarkdown scripts.
The first tool imports data and transforms data in the abundance table and metadata into an Rdata object. The second tool uses phyloseq and a custom R function to build two plots to visualize the composition of the sample. The third tool builds richness plots to visualize the alpha diversity of the sample. The fourth computes the beta distance matrixes and displays them on a heatmap. The fifth builds a heatmap plot and an ordination plot using ordination methods to visualize data structure. The sixth builds the clustering plot for cluster analysis with different linkage methods, and the seventh uses phyloseq and vegan functions in R to perform multivariate analysis of variance (MANOVA). As a preliminary step, users can obtain a phylogenetic tree based on OTUs using the FROGS Tree tool. This tool creates a multiple alignment of OTUs with Mafft and reconstructs phylogenetic tree with FastTree and phangorn R package.
fasta --input-biom filters. biom --out-fasta itsx. fasta --out-abundance itsx. biom --summary itsx. html --log-file itsx. log # Affiliation versus Unite affiliation_OTU. py --nb-cpus $CPU --reference $REFERENCE --input-biom itsx. biom -input-fasta itsx. fasta --output-biom frogs. biom --summary affiliation. html --logfile affiliation. log # Tabular abundance table biom_to_tsv. py --input-biom frogs. biom --input-fasta itsx. fasta --output-tsv frogs. tsv --output-multi-affi multiaff. tsv --log-file biom_to_tsv. log # Affiliation filter: parameters is very dependent on your dataset and questions. /affiliation_filters. py --input-biom frogs. biom --input-fasta itsx. fasta --output-frogs_affi_filter. biom --output-fasta frogs_affi_filter. fasta --summary frogs_affi_filter. html --impacted frogs_affi_filter_impacted_OTU.
TEST DATASET FOR ASSESSMENT OF THE FROGS ITS METHODOLOGY Our objective was to create datasets which make it possible, among other things, to test methods and tools to process data on fungi for metabarcoding. Consequently, we obtained computer-generated paired reads 2 x 250 nucleotides in length from artificial PCRs produced with Grinder software  [1]  on ITS1 or ITS2 from fungi which are as diverse as possible. Datasets are available on https://doi. org/10. 15454/AOT7UL. We extracted the reference sequences from the UNITE v7. 1 database centred on the eukaryotic nuclear ribosomal ITS region, we selected species without ambiguous key words i. e. without unknown and incertae key words in their name. UNITE 7. 1 from general FASTA releases contains 54 568 sequences composed of 8 179 and 46 389 sequences from respectively reference and representative species.
It is composed of 54 291 Fungi with 10 phyla, 52 classes and 185 orders. We produced simulated data reflecting this distribution. The sequences used to support artificial amplification by in silico PCR were taken from the UNITE v7. 1 database thanks to the python script treeSampling. py (https://github. com/geraldinepascal/FROGS/tree/master/assessment/bin) and see supplemental Figure  1  that describe the following procedure. We picked 35, 115, or 515 species of fungi to create three different types of diversity. We used Grinder, which is, among other things, a programme designed to create random amplicon sequence libraries based on DNA reference sequences. We produced paired-end reads R1 and R2, type Illumina Miseq. We incorporated sequencing errors. Reads were simulated by increasing the number of substitutions and indels linearly along the reads (Grinder parameters: -mutation_dist linear 2.
54e-1 2. 79e-1 and -mutation_ratio 98. 6 1. 4). We generated samples with two types of abundance distribution that follow either a power law or a uniform law. For each type of diversity, we merged these first datasets with exclusive datasets of chimera (3 types of chimera -Grinder parameters: -chimera_dist 314 38 1) to obtain an overall chimera rate of 2% per dataset. We focused on including reads from ITS1 and ITS2 of long lengths, not overlaid by two paired 250-nucleotide reads (min:132 nucl. to max 763 nucl. c. f. sections #Amplicon length distribution and #List of non-mergeable amplicons available from http://frogs. toulouse. inrae. fr/ITS/frogs-its. html#35_species, #115_species, #515_species). Thus, the resulting paired-end reads become unmergeable using sequence merging tools such as FLASH, VSEARCH or PEAR and reflect the reality of the sequences obtained in vitro.
Negative and True Positive rates) are calculated in relation to the expected. Each tool was compared to FROGS using a Wilcoxon signed-rank test, ****: p <= 0. 0001; ***: p <= 0. 001; **: p <= 0. 01; *: p <= 0. 05; ns: p > 0. 05). Affiliations and associated abundances are taken into account. (N. B. all results on ITS2 datasets are comparable). See complete results on: http://frogs. toulouse. inrae. fr/ITS/frogsits. html#Metrics_calculated_in_relation_to_the_expected19 Boxplots of the results of the processing of the synthetic datasets (515 species, ITS1, uniform abundance distribution) of the 4 tools tested (DADA2 paired-end (pe), DADA2 single-end (se) (pale and dark purple), FROGS (bluish green), QIIME-pe, QIIME-se (pale and dark khaki) and USEARCH (cyan)). Four metrics (Abundance divergence, False Positive, False Negative and True Positive rates) are calculated in relation to the expected.
Each tool was compared to FROGS using a Wilcoxon signed-rank test, ****: p <= 0. 0001; ***: p <= 0. 001; **: p <= 0. 01; *: p <= 0. 05; ns: p > 0. 05). Affiliations and associated abundances are taken into account. (N. B. all results on ITS2 datasets are comparable). See complete results on: http://frogs. toulouse. inrae. fr/ITS/frogsits. html#Metrics_calculated_in_relation_to_the_expected28.
ACKNOWLEDGMENTS The authors are grateful to the genotoul bioinformatics platform Occitanie (doi:  10. 15454/1. 5572369328961167E12 )and  Migale bioinformatics facility  (doi:  10. 15454/1. 5572390655343293E12 )for providing help, computing and storage resources. The authors thank  Valentin Marcon ,  Patrick Durand ,  Laure Quintric  and  Olivier Inizan  for the packaging of FROGS under CONDA. The authors thank  Ta Thi Ngan  for her involvement in the development of FROGSSTAT tools. The authors also thank the members of the  METABARFOOD  project for allowing us to use the data generated as part of the project, particularly  Monika Coton ,  Jérôme Mounier  and  Audrey Pawtowski  who were in charge of the fermented meat samples. The authors are grateful to  Daphne Goodfellow  for attention to the English-language version.
FalseSupplemental figure 5 : 5 Negative and True Positive rates) are calculated in relation to the expected. Each tool was compared to FROGS using a Wilcoxon signed-rank test, ****: p <= 0. 0001; ***: p <= 0. 001; **: p <= 0. 01; *: p <= 0. 05; ns: p > 0. 05). Affiliations and associated abundances are taken into account. (N. B. all results on ITS2 datasets are comparable). See complete results on: http://frogs. toulouse. inrae. fr/ITS/frogsits. html#Metrics_calculated_in_relation_to_the_expected19 Boxplots of the results of the processing of the synthetic datasets (515 species, ITS1, uniform abundance distribution) of the 4 tools tested (DADA2 paired-end (pe), DADA2 single-end (se) (pale and dark purple), FROGS (bluish green), QIIME-pe, QIIME-se (pale and dark khaki) and USEARCH (cyan)). Four metrics (Abundance divergence, False Positive, False Negative and True Positive rates) are calculated in relation to the expected.
/github. com/geraldinepascal/FROGS-wrappers. FROGS supports ITS since version 3. 0 and is fully documented on the companion website http://frogs. toulouse. inrae. fr/. https:/FROGS: 7-STEP GUIDELINES FOR THE MANAGEMENT OF INTERNAL TRANSCRIBED SPACERS (ITS) Installing and updating FROGS are easy thanks to the FROGS repositories in CONDA (https://anaconda. org/bioconda/frogs) and Galaxy Toolshed (https://toolshed. g2. bx. psu. edu/view/frogs/frogs). Two github FROGS repositories are effective https://github. com/geraldinepascal/FROGS and.