dataesr/scientific-paragraphs-categorization huggingface dataset:
Moreover, Hypostomus is known to be a genera with low interspecific COI divergence  (de Queiroz et al. , 2020). Overall, results from our dataset conform to the now well-established notion that the 2% barcode gap is a good start to flag low inter-species divergence (only six pairs of species show a dNN lower than <2% in our dataset), but that it does not always apply to some groups of fish  (Pereira, Hanner, Foresti, & Oliveira, 2013; de Queiroz et al. , 2020)  and thus should not be interpreted as a stand-alone metric.
Conclusion The 1,284 COI sequences introduced in this study represent the first extensive fish barcode dataset of the Maroni and the first dataset of this scale for the Guiana Shield. Using barcoding as a secondary tool to identify specimens during this study has been a success. Although the majority of specimens could be identified on the field with morphologic and meristic methods only, this traditional way of identification has its limits: post-larvae or small damaged specimens are sometimes impossible to classify based on morphological keys  (Ward et al. , 2009). Species for which identification relies heavily on pigmentation (e. g. Moenkhausia and Jupiaba ) are also highly problematic when identification is performed long after sampling because conservation in ethanol and formaldehyde results in the loss of pigments from the specimens.
By building a DNA barcode library and relying on other sequences deposited by the scientific community, it was possible to easily verify some dubious identifications and pinpoint some overlooked diversity in our data. More than two thirds of the freshwater fishes of the Maroni now have at least one reference sequence available to facilitate identification. Biodiversity assessment in rivers have recently shown a trend of shifting from traditional "sampling by catch" methods to the less destructive use of environmental DNA metabarcoding  (Taberlet, Coissac, Hajibabaei, & Rieseberg, 2012) , a method that is currently being trialled in French Guiana rivers  (Cilleros et al. , 2019; Murienne et al. , 2019). This method relies on solid reference databases to identify environmental DNA sequences to the species level or below, and our dataset is a step forward towards this goal.
e. by using environmental DNA, would allow the application of this clustering landscape approach to compare levels of genetic diversity (as opposed to divergence) across the basin, another important indicator of the resilience capacity of species in an ecosystem. ) 2005 Table  3. Twenty-five species for which status has been updated during this study using COI barcodes, morphology, and geographic distribution. Species referred as "cryptic" are additions to the Maroni checklist and share occurrence with their nominal species. Species that have been flagged as putative re-identification are referred as "New Taxonomical Combination". For these, the nominal species are suspected not to be present in the Maroni basin. For eight putative species that are BIN-discordant on the BOLD database,  showing 83 sampling points (red dots) of specimens collected.
Acknowledgements We are grateful to  Francois Meunier ,  Yves Fermon , and  Philippe Keith ,  MNHN ;  Mark Sabaj  and  John Lundberg ,  ANSP ;  Philippe Gaucher ,  CNRS Guyane ;  Regis Vigouroux ,  Philippe Cerdan , and  Sebastien Le Reun ,  Hydreco Guyane ;  Chrystelle Delord ,  Marie Le Noc ,  Marie Nevoux  and  Jean-Marc Roussel ,  INRA ,  Rennes ;  Michel Jegu ,  IRD ;  Mael Dewynter ,  Benjamin Adam  and  Antoine Baglan ,  Biotope Guyane ;  Frederic Melki ,  Biotope France ;  Jan Mol ,  University of Suriname ;  Kenneth Wan Tong You  and  Paul Ouboter , NZCS;  Juan Montoya-Burgos ,  UNIGE ;  Sonia Fisch-Muller  and  Claude Weber , MHNG;  Raphaelle Rinaldo  and  Guillaume Longin ,  Parc Amazonien de Guyane ;  Gregory Quartarollo  and the  Guyane Wild Fish Association  ;  Olivier Tostain ,  Ecobios Cayenne ;  Jonathan W.
Intuitively, the quality of a patch representation plays a major role in determining the eventual learning outcome. In this paper, to boost the effectiveness of existing solutions that employ the properties of patches, we wish to learn vector representations of the code changes in patches that can be used for a number of tasks. We propose a new deep learning architecture named CC2Vec that can effectively embed a code change into a vector space where similar changes are close to each other. As log messages, written by developers, are used to describe the semantics of the code change, we use them to supervise the learning of code changes' representations from patches. Specifically, CC2Vec optimizes the vector representation of a code change in a patch to predict appropriate words, extracted from the first line of the log message.
Next, we employ multiple comparison functions to capture the difference between two embedding vectors representing removed and added code. This produces features representing the relationship between the removed and added code. Each comparison function produces a vector and these vectors are then concatenated to form an embedding vector for the affected file. Finally, the embedding vectors of all the affected files are concatenated to build a vector representation of the code change in a patch. After training is completed, CC2Vec can be used to extract representations of code changes even from patches with empty or meaningless log messages (which are common in practice  [26, 38, 39] ). CC2Vec is also programminglanguage agnostic; one can use it to learn vector representations of code changes for any language.
For the task of identifying bug fixing patches, CC2Vec helps to improve the best performing baseline by 5. 22%, 9. 18%, 4. 36%, and 6. 51% in terms of accuracy, precision, F1, and Area Under the Curve (AUC). For just-in-time defect prediction, CC2Vec helps to improve the AUC metric by 7. 03% and 7. 72% on the QT and OPENSTACK datasets  [42]  as compared to the best baseline. The main contributions of this work are as follows: • We propose a deep learning architecture, namely CC2Vec, that learns distributed representations of code changes guided by the semantic meaning contained in log messages. To the best of our knowledge, our work is the first work in this direction. • We empirically investigate the value of integrating the code change vectors generated by CC2Vec and feature vectors used by state-of-the-art approaches on three tasks (i.
Preprocessing The code change of the given patch includes changes made to one or more files. Each changed file contains a set of lines of removed code and added code. We process the code change of each patch by the following steps: • Split the code change based on the affected files. We first separate the information about the code change to each changed file into a separate code document (i. e. , File 1 , File 2 , etc. , see Figure  1 ). • Tokenize the removed code and added code lines. For the changes affecting each changed file, we employ the NLTK library  [9]  for natural language processing (NLP) to parse its removed code lines or added code lines into a sequence of words. We ignore blank lines in the changed file. • Construct a code vocabulary. Based on the code changes of the patches in the training data, we build a vocabulary V C.
Task 1: Log Message Generation 3. 1. 1 Problem Formulation. While we learn representations of code changes with the aid of log messages, we also study the task of generating log messages from code changes. Developers do not always write high-quality log messages. Dyer et al. [16]  reported that around 14% of log messages in 23,000 Java projects on SourceForge 2 were empty. Log messages are important for program comprehension and understanding the evolution of software, therefore this motivates the need for the automatic generation of log messages. In this task, given the code change of a given patch, we aim to produce a brief log message summarizing it.
State-of-the-art Approach. The state-of-the-art approach is NNGen  [39] , which takes as input a new code change with an unknown log message and a training dataset (patches), and outputs a log message for the new code change. NNGen first extracts code changes from the training set. Each code change in the training set and the new code change are represented as vectors in the form of a "bag-of-words"  [41]. NNGen then calculates the cosine similarity between the vector of the new code change and the vector of each code change in the training set, and selects the top-k nearest neighbouring code changes in the training dataset. From these k nearest neighbours, the BLEU-4 score  [48]  is computed between each of the code changes in the top-k and the new code change with an unknown log message.
Experimental Setting. The purpose of evaluating CC2Vec on this task is to determine if the code change representations received from CC2Vec outperform the naive representation used by Liu et al. [39]. Jiang et al. [26]  originally collected and filtered the commits to construct the original dataset. Another version of the dataset was used by Liu et al. [39] , who modified the original dataset. Jiang et al. extracted a total of 2 million patches from the 1K most starred Java projects. They collected the first line of each log message. To normalize the dataset, patch ids and issue ids were removed from the code changes and log messages. Patches were filtered to remove merges, rollbacks, and patches that were too long. The log messages that do not conform to verb-direct-object pattern, e. g. "delete a method", are also removed.
After filtering, the dataset contains 32K patches. Still, even with all this cleaning, Liu et al. [39]  investigated the dataset and found that there were many patches with bot messages and trivial messages. Bot messages refer to messages produced automatically by other development tools, such as continuous integration bots. Trivial messages refer to messages containing only information that can be obtained by looking at the names of the changed files (e. g. "modify dockerfile"). Such messages are of low quality and Liu et al. used regular expressions to locate and remove these patches. We used the original dataset of Jiang et al. [26]  and the cleaned dataset of Liu et al. [39]  for evaluation. While the original dataset consists of a training dataset of 30K patches and a testing dataset of 3K patches, the cleaned dataset consists of a training dataset of 22K patches and a testing dataset of 2.
Task 2: Bug Fixing Patch Identification 3. 2. 1 Problem Formulation. Software requires continuous evolution to keep up with new requirements, but this also introduces new bugs. Backporting bugfixes to older versions of a project may be required when a legacy code base is supported. For example, Linux kernel developers regularly backport bugfixes from the latest version to older versions that are still under support. However, the maintainers of older versions may overlook relevant patches in the latest version. Thus, an automated method to identify bug fixing patches may be helpful. We treat the problem as a binary classification problem, in which each patch is labelled as a bug-fixing patch or not. Given the code change and log message, we produce one of the two labels as the output.
Our Approach. CC2Vec is first used to learn a distributed representation of code changes on the whole dataset. All patches from the training and test dataset are used since the log messages of the test dataset are not the target of the task. Next, we integrate these vector representations of the code changes with the two existing approaches. To use CC2Vec in PatchNet, we concatenate the vector representation of the code change extracted by CC2Vec with the two embedding vectors extracted from the log message and code change by PatchNet to form a new embedding vector. The new embedding vector is fed into PatchNet's classification module to predict whether a given patch is a bug fixing patch. For the approach proposed by Tian et al. [57]  which uses an SVM as the classifier, we pass the vectors produced by CC2Vec from the code change into the SVM as features.
Experimental Setting. The goal of this task is to investigate if CC2Vec helps existing approaches to effectively classify bug-fixing patches. We use the dataset of Linux kernel bug-fixing patches used in the PatchNet paper. This dataset consists of 42K bug-fixing patches and 40K non-bug-fixing patches collected from the Linux kernel versions v3. 0 to v4. 12, released in July 2011 and July 2017 respectively. Patches in this dataset are limited to 100 lines of changed code, in line with the Linux kernel stable patch guidelines. The nonbug-fixing patches are selected such that they have a similar size, in terms of the number of files and the number of modified lines, as the bug-fixing patches. Following the PatchNet paper, we use 5-fold cross-validation for the evaluation. To compare the performance of the approaches, we employ the following metrics: • Accuracy: The ratio of correct predictions to the total number of predictions.
JIT defect prediction tools provide early feedback to software developers to optimize their effort for inspection, and have been used at large software companies  [44, 51, 55]. We model the task as a binary classification task, in which each patch is labelled as a patch containing a defect or not. Given a patch containing a code change and a log message with unknown label, we label the patch with one of the two labels. 3. 3. 2 State-of-the-art Approach. The state-of-the-art approach is DeepJIT, proposed by Hoang et al. [21]. DeepJIT takes as input the log message and code change of a given patch and outputs a probability score to predict whether the patch is buggy. DeepJIT employs a Convolutional Neural Network (CNN)  [31]  to automatically extract features from the code change and log message of the given patch.
However, DeepJIT ignores information about the structure of the removed code or added code, instead relying on CNN to automatically extract such information. [42]. The QT dataset contains 25K patches over 2 years and 9 months while the OPENSTACK dataset contains 12K patches over 2 years and 3 months. 8% and 13% of the patches are defective in the QT dataset and the OPENSTACK datasets respectively. Like Hoang et al. [21] , we use 5-fold cross validation for the evaluation. To compute the effectiveness of the approaches, we use the Area Under the receiver operator characteristics Curve (AUC), similar to the previous studies.
Results. The evaluation results for this task are reported in Table  3. The use of CC2Vec with DeepJIT improves the AUCS score of DeepJIT, from 76. 8 and 75. 1 to 82. 2 and 80. 9 on the QT and OPENSTACK datasets respectively. Specifically, CC2Vec helps to improve the AUC metric by 7. 03% and 7. 72% for the QT and OPENSTACK datasets, respectively, as compared to DeepJIT. This indicates that CC2Vec is effective in learning a useful representation of patches that an existing state-of-the-art technique can utilize.
Ablation Study Our approach involves five comparison functions for calculating the difference between the removed code and added code. To estimate the usefulness of comparison functions (see Section 2. 4. 2), we conduct an ablation study on the three tasks: log message generation, bug fixing patch identification, and just-in-time defect prediction. Specifically, we first remove the comparison functions entirely and then remove these functions one-by-one. For each task, we compare the CC2Vec model and its six reduced variants: All-all (omit all comparison functions), All-NT (omit the neural network tensor comparison function), All-NN (omit the neural network comparison function), All-sim (omit the similarity comparison function), All-sub (omit the subtraction comparison function), and All-mul (omit the multiplication comparison function).
In contrast, we learn representations of code changes, which can contain modifications to multiple different functions, across multiple files. Several of the models related to code changes' representation were discussed in Section 3. These models often do not model the hierarchical structure of a code change or require handcrafted features that may be specific to a single task  [5, 26, 28-30, 39, 44, 57, 60]. Two techniques using deep neural networks, PatchNet  [22]  and DeepJIT  [21] , are most similar to our work. However, as discussed earlier, our work differs from theirs in various ways. A fundamental difference is in the generality of the techniques. CC2Vec is not specific to a single task. Rather, CC2Vec can be trained for multiple tasks, including both generative and classification tasks.
[39]ormance of each approach on the original and cleaned dataset reported in BLEU-4 Our Approach. To use CC2Vec for this task, we propose LogGen. Similar to the nearest neighbours approach used by Liu et al. [39], LogGen reuses and outputs a log message from the training set. However, instead of treating each code change as a bag of words, LogGen uses code change vectors produced by CC2Vec. CC2Vec is first trained over the training dataset. Given a new code change from the test dataset with an unknown log message, we find the code changes with a known log message that have the closest CC2Vec vector. Like Liu et al.
3. 1. 5Results. We report the performance of LogGen, NNGen and NMT in Table 1. LogGen outperforms both NNGen and NMT. The Clean dataset refers to the dataset which Liu et al. filtered out patches with bot and trivial log messages. On this dataset, LogGen outperforms NNGen and NMT by a BLEU-4 score of 4. 06 and 6. 29 respectively. LogGen improves over the performance of NNGen by 24. 75%, a greater improvement than NNGen's improvement over NMT of 15. 70%. On the original dataset collected by Jiang et al. , LogGen outperforms NNGen and NMT by a BLEU-4 score of 4. 65 and 11. 28. These results indicate that LogGen can improve over the performance of NNGen and NMT by 12. 06% and 2. 07% in terms of the BLEU-4 score respectively.
3. 3. 3OurApproach. Similar to the previous task (i. e. , bug fixing patch identification), CC2Vec is first used to learn distributed representations of the code changes in the whole dataset. All patches from the training and test dataset are used since the log messages of the test dataset are not part of the predictions of the task. We then integrate CC2Vec with DeepJIT. To use CC2Vec with DeepJIT, for each patch, we concatenate the vector representation of the code change extracted by CC2Vec with two embedding vectors extracted from the log message and code change of the given patch extracted by DeepJIT to form a new embedding vector. The new embedding vector is fed into DeepJIT's feature combination layers, to predict whether the given patch is defective. 3. 3. 4ExperimentalSetting. The purpose of this task is to evaluate if CC2Vec can be used to augment existing approaches in effectively classifying defective patches.
A flexible mechanism for obtaining and renewing consent for data use and sharing is required that provides appropriate and meaningful incentives for users to capitalize from data sharing and ensures transparency for users to be aware of which of their dataset has been accessed, by whom, for what purpose and under what conditions. We address all these issues (security, privacy, usertransparency and control, and incentives for data sharing) by proposing a new platform for user modeling using distributed ledgers and smart contracts, relying on user-controlled privacy and data-sharing policies encoded in smart contracts. It also supports creating incentives for users to share their data in terms of rewards (micro-payments or credits) encoded in smart contracts. Thus, users become owners of their data and can decide how their data is collected and used, as well as shared, and benefit not only in terms of improved personalized experience with the service but also directly, for example, by participating in the share of the advertising revenue generated by the service provider.
MultiChain, being a private blockchain, has the potential to replace traditional centralized databases used to store user data in a decentralized way offering more cryptographic auditing features. Optionally we can store any published item off-chain that saves storage place and bandwidth. MultiChain, however, due to its current architecture, cannot support an access control mechanism, which the proposed platform needs, to provide users means to control how their data is shared and the rewards they get. Therefore, we use Ethereum, which supports smart contracts and commits the contracts' transactions. The smart contracts govern accountability of access and provide incentives to the users and application owners for sharing user data. Combining the security and immutability of data stored in the blockchain, with the specific strengths of two popular blockchains-MultiChain and Ethereum-allows us to combine their advantages: proper data storage and data sharing and smart contracts for access control mechanisms.
The blockchain software pattern represents a digital ledger-a database with an immutable record of every transaction which has ever taken place. Each block aggregates a timestamped batch of transactions to be placed in the chain. There is also a cryptographic signature to identify each block. Each block refers to the signature of the previous block in the chain, and that chain can be traced back to the very first (Genesis) block created in the chain. The key idea is that there is no centralized authority to determine what is true or what is false; instead, multiple distributed parties come to a consensus that is entered into the ledger and after that can be accessed by anyone  (Shrestha and Vassileva, 2016). Computationally, it is impracticable for anyone (or less than a majority consensus) to go back and alter the history because the blockchain represents a chronological chain of events.
MultiChain MultiChain is a private permissioned blockchain that provides the privacy and control required in an easy to configure and deploy package  (Greenspan, 2013). It supports UNIX and Windows servers and produces a rich JSON-RPC API for easy integration with existing systems. Unlike any other blockchains, MultiChain solves the problems of privacy and openness via integrated management of user permissions with its three main objectives  (Greenspan, 2013) : 1. To enable only the chosen participants to see the blockchain's activity, 2. To ensure that only selected transactions are permitted, 3. To securely conduct mining without proof of work and its associated costs. MultiChain being a closed system allows participants to control the maximum block size and set all of the blockchain's parameters in a configuration file, thereby resolves the problems of scalability.
For deployment and testing of smart contracts on Ethereum blockchain, a truffle framework 4  can be used since it has built-in smart contract compilation, linking, deployment and binary management. As of now, our smart contract runs on EVM. However, with the implementation of the Ethereum Web Assembly (EWASM) 5  in the future, smart contract development can be done in any other programming languages besides Solidity, and that will also speed up the function call between Web Assembly and JavaScript (JS).
Users can back up (download) their data, leave one service and try out new services whenever they want. The DTP offers an open-source service-to-service data portability platform 7  that enables users to move their data between online service providers. The contributors to the project have felt that interoperability is central to innovation for data portability. Amazon also has OpenDataRegistry 8  to allow the public to provide/upload and discover/access useful datasets via AWS resources. The datasets to be shared must be documented, actively maintained, and supported by data providers since they are not provided by AWS but are curated or optimized for analysis using AWS tools. Datasets are included at the discretion of the AWS Open Data team, which may remove datasets from the registry at any time.