dataesr/scientific-paragraphs-categorization huggingface dataset:
,or +) is equal to "bot", "ghbot", "bors", "travis" or "bot". A few remaining email strings without "@" were discarded, like "localhost", "N/A" or empty string. Manual inspection of the most active 100 emails without a GitHub account revealed a few more bots. The list of manually discarded bots is available in the file botlist. csv. Our dataset includes the bots among the users, flagged with a Boolean "is_bot" attribute to enable filtering.
Data Records We host our data on Figshare  38  and the code used to collect and process the data from our sources on GitHub (https://github. com/wschuell/repo_datasets). Both platforms track the history of the data and code, allowing researchers to use any version they prefer as we continue to update and extend both. We share data in several formats, noting that in no format does the data exceed 6 GB when compressed. Before we describe the data, we discuss data pseudonymisation. To preserve developer privacy, we provide data that is scrubbed of information that can directly be used to identify individuals. We do so in the following way: we discard name attributes and hash (via MD5 with a random salt) email addresses and GitHub/ GitLab logins. Researchers interested in studying social or demographic characteristics of developers, such as gender  39, 40  , geography  [41] [42] [43]  , or both  44  , could adapt our approach to data collection and analyse these attributes.
technical Validation In this section, we report statistics on the completeness of our data. An advantage of defining the Rust ecosystem as all those packages hosted on Cargo, is that we can precisely measure how many packages we can successfully integrate into our database. In particular, we can report the share of packages that we can connect to repos on the social coding platforms GitHub and GitLab. As we will see below, we have a very high rate of linkage. Moreover, the packages that we could not integrate are typically those with very few downloads. This suggests that most projects on Cargo that do not appear on GitHub or GitLab are small personal projects or preliminary work. Package coverage among repositories. Some Rust packages hosted on Cargo could not be linked to repos on GitHub or GitLab.
Across the cloned repos, we gathered 5,656,407 commits, the elemental units of contribution in the git version control system. Excluding 219 bots identified among the GitHub accounts, these contributions were made by 58,329 GitHub users and 450 GitLab users. The raw data contains 89,399 identifying email addresses, highlighting the significant amount of disambiguation of author identities our pipeline implements. 14,266 of them could not be associated to a GitHub or GitLab account. comparison with other data sources. We first compare our data with data collected in the GHTorrent project  15. The GHTorrent project aims to collect all activity on GitHub for use in research. As we have already noted, most activity in the Rust ecosystem takes place on GitHub, with a small but significant share taking place on GitLab.
Besides the inclusion of GitLab data, we observed that our data contains a significant amount of activity hosted on GitHub that is missing from the GHTorrent database (the SQL version), when comparing the data in our dataset tagged as happening before the last date of user creation in the GHTorrent database-May 31st 2019, just before midnight-and corresponding to the repositories that could be cloned. Specifically, we found only 14,563 unique users (vs. 21,989 GitHub users in our database), 13,623 unique repos (vs. 16,069 identified GitHub repos in our database), and 1,299,414 unique commits (vs. 1,942,997 GitHub hosted commits in our database before the last date of GHTorrent). The GHTorrent project uses the GitHub REST API and collects data from the public event timeline using user-donated API keys.
Outages on either the GHTorrent side or on the GitHub REST API, or rate limited API keys may explain missing data. While GHTorrent remains an excellent source of dataset for all of GitHub, these comparisons suggests that for a focused look at a single ecosystem, a customised pipeline can significantly increase data coverage. More detailed statistics concerning the comparison can be found on Figshare  38  in the file ghtorrent_comparison. yml. We also note that our data collection pipeline is not the only way to collect similar data on OSS ecosystems. For example, the GrimoireLab toolchain is a collection of tools to gather and analyze data on software  46. These tools provide users with sophisticated analyses of the health and activity levels of particular projects, and groups of projects.
In particular its Perceval data retrieval module can do many of the same things as our collection pipeline. Our tool focuses rather on one thing: collecting a nearly complete dataset on the Rust ecosystem quickly (i. e. maximizing GraphQL API calls vs. REST API calls), with minimal requirements for users seeking to replicate or refresh the dataset. In this sense the two can be considered as potential complements, rather than substitutes: an analyst studying specific libraries in the Rust ecosystem can easily use the GrimoireLabs to obtain metrics on those libraries and additional data about them, for example from Jira or Twitter.
Usage Notes To demonstrate how to read in and analyse the database, we provide short Jupyter notebooks that extract data and carry out elementary data manipulations, with the data aggregated at the monthly level. These notebooks are included with the other software in our materials. In one, we create the dependency network of the Rust ecosystem at different times and measure its growth. This data can be used to study ecosystem health, as errors and issues are known to spread through these networks  6. In another, we plot the time series of downloads and new pull requests to specific repositories over time. Such time series can be used to study the dynamics of collaboration, use, and success at a fine-grained level  8. In a third, we show how to load in the data of developers and packages as a rectangular matrix, which can be analysed as a bipartite network or, after a projection, as a developer-developer collaboration network.
The bipartite network could be used to study the overall complexity of the ecosystem  47, 48  , while the developer network reveals patterns of collaborations between projects  49. In Fig. 3  we present several illustrative examples of descriptive analyses resulting from the demonstration notebooks. Our dataset can also be used as an input to study current themes of the empirical software engineering community. For example, our data can extend in time the work of Decan et al. 6  on transitive dependencies and library centrality in Rust. These measures themselves provide interesting ways to quantify success and importance of libraries. Another interesting area of research that our data might be applied to is the concept of code smells, in particular community smells  50. Code smells are problematic kinds of patterns in software, while community smells refer to such sub-optimal patterns on the social and collaborative levels of collective software development.
For example, Tamburri et al. 50 describe several community smells such as the "lone wolf " -when a single developer acts in a unilateral and inconsiderate way -or the "organizational silo" -when developer teams working on different parts of a codebase only communicate through one or two team members. These smells are quantified in part by considering the collaboration and communication networks of developers. Our dataset can be used to calculate many of these concepts including collaboration network measures and socio-technical congruence  9  at the ecosystem level by considering which developers contribute to which libraries and their interactions in Fig. 2  Data coverage: we check the number of Rust packages for which we could identify and download a corresponding Git repo (from GitHub or GitLab) in terms of their use, measured in downloads.
These include metrics of community support and adoption (number of contributors, issue and pull request response times) and maturity (releases, number of forks, age), and to some extent quality (code size and rate of issue resolution). More generally, that is beyond the broad research field of empirical software engineering, our dataset can be used to explore the interactions between social collaboration, technical dependencies, and the visibility and usage of components of a large software system. The dynamic interactions between these layers of the data offer significant potential for research relating to massive decentralized cooperation (similar to Wikipedia  52, 53  ), the dynamics of teams and their success in digital communities  21  , and the evolution of software systems  54.
It is called vouchering; the act of depositing strains and their related metadata in a recognized culture collection. Many journals require vouchering in order to publish, so you will have encountered this practice at some point, but as Editor in Chief for Application in Plant Sciences laments, authors keep asking, 'Why do vouchers even matter?' [8]. Specimens have a myriad applications  [9] , but in this paper we offer a perspective for plant pathologists and regulatory scientists working with phytopathogenic bacteria in particular. We plan to convince you that all your hard work isolating and purifying should be captured in long-term storage at registered and dedicated facilities, for both your own benefit and for scientists to come. Below is a series of case studies based on several important and potentially destructive plant pathogenic bacteria that demonstrate why the phytobacteriology community should play an active role in depositing important research isolates into national and/or international culture collections.
citri was imported, stored, and accessioned in the DAR collection. This collection consists of 32 global isolates, seven Australian incursion strains, and three international border intercepts. This collection ranged in dates from 1988 to 2018, but it wasn't accessioned until 2020. The origins of these isolates include the USA, Oman, Saudi Arabia, Iran, Thailand, Torres Strait, East Timor, and South-East Asia. There are whole-genome data for the isolates in this collection, which were used to develop species-specific diagnostic assays for the detection of Xanthomonas citri subsp. citri in Australian conditions. This assay was incorporated into an updated National Diagnostic Protocol (NDP-9 Asiatic citrus canker Xanthomonas citri subsp. citri V1. 2), which was submitted at the end of March 2018.
There was only a single South African Xvv isolate that had this prophage gene cluster, but unfortunately, although the genome sequence is publicly available in GenBank, the isolate was not deposited, and the strain no longer exists. The importance and origin of this prophage region to the increased virulence of Xvv isolates from the U. S. and Argentina is still unclear. Depositing strains of pathogenic species that cause common diseases, even if thought to be of minor economic significance, will continue to be of great importance to understanding the next emerging pathogen, as global trade and movement of seed continue to expand. In addition, while genome sequence data can be safely stored on hard drives and backed up in cloud-based storage, there are limitations to the inferences that can be drawn from these data if the biological organism from which they were generated is not also 'backed up' in a culture collection.
We propose that it be treated much like the sequence and wholegenome data generated from many of these strains. Specimens should be deposited in registered culture collections in the same way sequence data is deposited in registered databases, such as GenBank http://www. ncbi. nlm. nih. gov/genbank/, accessed on 27 March 2022, DNA Data Bank of Japan (DDBJ) (http://www. ddbj. nig. ac. jp/index-e. html, accessed on 27 March 2022), and EMBL Nucleotide Sequence Database (EMBL-Bank) (http://www. ebi. ac. uk/embl/, accessed on 27 March 2022). Below we provide a list of best practices that researchers, scientific societies, scientific journals, funding agencies, and federal governments should consider for their laboratories, research projects, funding programs, and publishing guidelines. Develop a specimen management plan.
This is already included in standards for the development of diagnostic tools (for instance, EPPO 2020  [46] ). Additionally, journals can require this of authors, rather than it being merely a suggestion. Researchers are strongly advised to deposit strains in advance, so that they can use culture collection accession numbers in their publications, improving future consistency. Collect, record, and safeguard specimen metadata. When depositing new strains, provide the collection with as much metadata as possible associated with the strains. This provides a centralized location for all metadata associated with a strain, increasing the biological value of the strain for future diagnostic and research purposes. Moreover, for regulation purposes, some information is crucial, such as the country of origin (in general, this is the country of isolation) or the funding organization of the isolation.
Nagoya protocol) and will help research organizations to comply with international regulations. Make microbial specimens available to the scientific community. Journals should require that strains used for a new disease report, associated with a publicly available or published whole-genome sequence or used in the development of diagnostic protocols, be deposited in at least one culture collection. This is already included in standards for In general, we strongly encourage scientists to deposit their important resources in public culture collections, where they will be made available long term for the benefit of the whole scientific community. Because the quality of the resources is dependent on the quality of the associated data, depositors are also strongly encouraged to transmit the most accurate and complete data, even if these data do not appear to be important at the time of deposit.
Conclusions The significance of existing and yet-to-be-discovered bacterial plant pathogens is shown in this paper. Their importance will increase as the world undergoes inevitable climatic changes. As host plants become established in new territories, there will be new interactions with the pathogenic bacteria that are already present, and pathogens will be spread into new geographical ranges. The problem is that there is a need to accelerate the rate at which data are collected and used to answer the questions the future will bring but the number of depositions of plant pathogenic bacteria in the relevant collections is actually decreasing (Figure  3 ). Further, as has been outlined, we do not start from the best of positions: bacteria are among the most diverse and abundant organisms on and in the earth, yet they are extremely poorly represented in the world's collections.
Conclusions The significance of existing and yet-to-be-discovered bacterial plant pathogens is shown in this paper. Their importance will increase as the world undergoes inevitable climatic changes. As host plants become established in new territories, there will be new interactions with the pathogenic bacteria that are already present, and pathogens will be spread into new geographical ranges. The problem is that there is a need to accelerate the rate at which data are collected and used to answer the questions the future will bring, but the number of depositions of plant pathogenic bacteria in the relevant collections is actually decreasing (Figure  3 ). Further, as has been outlined, we do not start from the best of positions: bacteria are among the most diverse and abundant organisms on and in the earth, yet they are extremely poorly represented in the world's collections.
The ultimate question is, 'How does a bacterial plant pathogen interact with its host(s) in differing conditions?', and sequence data might never be able to tell us that. We have outlined in our list of best practice recommendations the importance of journals and funding agencies requiring depositions of strains into public collections, the importance of bodies responsible for diagnostic assay development and researchers utilizing publicly accessible cultures from collections, and the importance of culture deposits being accompanied by extensive metadata. We also ask that enough time be allowed when transferring privately held collections to a public collection. The benefits are highlighted in Box 1-both for you and for the scientific community. In the introduction to this review, we asked some questions.
gbif. org/ (accessed on 27 March 2022)), or WDCM the World Data Center for Microorganisms (http://refs. wdcm. org/ (accessed on 27 March 2022)). In Europe ECCO, the European Culture Collections' Organisation regroups the majority of European microbial collections (https://www. eccosite. org/ (accessed on 27 March 2022)). In Europe, the Microbial Resource Research Infrastructure MIRRI (https://www. mirri. org/ (accessed on 27 March 2022)) is being built, and a general catalogue, along with other services, is already available. Table  2  below lists culture collections specialized in plant-pathogenic bacteria. Through these collections, the data on cultures cited in this article are publicly available.
://www. dpi. nsw. gov. au/about-us/services/collections/collection-servicesICMP New Zealand https://www. landcareresearch. co. nz/icmpNCPPB United K. B. was funded by the  United States Department of Agriculture-Agricultural Research Service. B. S. W. was supported by funding from the  New Zealand Ministry of Business and Innovation (MBIE) Infrastructure Programme of the Strategic Science Investment Fund (SSIF)   C09X1703  through the  Biota Portfolio of Manaaki Whenua-Landcare Research. T. C. and J. B. were supported by the  NSW Department of Primary Industries, Australia. P. P. was supported by  INRAE ,  French National Research Institute for Agriculture, Food and Environment. A. A. was funded by the  UK Department for Environment, Food & Rural Affairs (Defra). Institutional Review Board Statement: Not applicable.
Initial Approach: Establishing Possible EHT-PHT Biomarkers Table  1  summarizes the characteristics of all patient samples analyzed. Briefly, our final dataset consisted of 231 plasma samples collected from patients with EHT (104 PA + 94 PPGL + 33 CS) and 106 samples from patients with PHT. Results from the peak picking procedure are shown in Figure  1. Results from the peak picking procedure are shown in Figure  1. The Principal Component Analysis (PCA) score plot of the complete dataset including all 86 peaks (Figure  S1 ) showed Quality Control (QC) samples clustering closely The Principal Component Analysis (PCA) score plot of the complete dataset including all 86 peaks (Figure  S1 ) showed Quality Control (QC) samples clustering closely together, indicating a low analytical variation compared to the sum of biological and preanalytical variation.
The result shown in Figure  2a  appeared promising for the successful separation of the two main disease groups (EHT and PHT), as the two disease groups resulted in slightly different scores along the second principal component (PC2). The classification analyses of these datasets resulted in high accuracies of approximately 80% when attempting to distinguish PHT from EHT forms (Table  S2  and Table 2 ). Lower balanced accuracies (e. g. , 65%, Table  2 ) resulted from separating all groups from each other. Table  3  depicts the signatures obtained from scenario EHT-PHT. ITPD (GYLU), München (GYMU), Würzburg (GYWU), Torino (ITTU3), Nijmegen (NLNI) and Warsaw (PLWW), samples in cluster Paris (FRPA1) PHT had low glutamine and high glutamate, whereas samples from centers FRPA1 (non-PHT), FRPA2, Glasgow (GBGL2), Galway (IRGA) and Padova (ITPD/ITPD3) were characterized by high lactate and ornithine, GBGL2 samples were further characterized by high glutamate and low glutamine, and ITPD/ITPD3 samples had high methanol.
ASCA correction was not possible on the CS-PHT or the all vs. all dataset, as there were no samples represented by cluster 1 in the CS group. According to Table  3 , glycine and lactate were the differences between the ASCA and Initial Approach signatures of scenario EHT-PHT, indicating that their levels were normalized by ASCA, diminishing their importance. In the other scenarios, ASCA similarly normalized levels of metabolites that correlated to the first principal component of Figure  2.
Correcting for Confounders: Approach B (Metabolite Exclusions) As shown in Figure  2 , center and sample age appeared to be major sources of variation in our data. Table  S4  provides an overview of all analyses carried out to assess the effects of these confounders on the various datasets. Overall, accuracies were higher when comparing groups defined by confounders than they were based on disease groups (Initial Approach). We employed PLSDA to establish the signatures of these effects (Table  4 ) and compared cluster 1 (excluding the FRPA1 PHT group due to a distinct signature) to cluster 2, as well as samples with a sample age below the median to those with a sample age above the median (calculated including all samples). We used PCA, due to limited sample sizes, to compare the FRPA1 PHT group to the rest of cluster 1.
Metabolites acetylcarnitine, creatine, dimethyl sulfone, glucose, glutamate, glutamine, glycerol, glycine, lactate, methanol, methionine, ornithine, pyruvate and the unknown signal at 3. 284 ppm were all found to be related to these confounders. In relation to disease groups, all scenarios resulted in signatures that included confounder-related metabolites. Notably, 9/13 metabolites making up the EHT-PHT signature depicted in Table  3  were also found to be related to a confounder. Analyses of data after we excluded confounder-related peaks yielded lower accuracy estimates than those calculated from the complete datasets (Table  S5  and Table 2 ). Regarding signatures, Approach B resulted in a unique set of metabolites compared to the other approaches (with 8/12 metabolites not found with any other approach in the EHT-PHT scenario, Table  3 ).
We expected this approach to yield the most reliable signature from our dataset and research questions, as it was the only one that both provided a complete view of the NMR plasma metabolome and kept confounding effects to a minimum by excluding compromised samples. Even so, we do not claim that Approach C signatures have merit in clinically discriminating PHT from any EHT form, as these analyses were restricted to certain centers and so included far fewer samples than originally planned. Approach C models predicted excluded PHT sample groups accurately, but sensitivity was low, while the highest accuracy achieved was 65% (PPGL-PHT model). Moreover, analyses comparing centers GYDR to ITTU3 were more accurate than those comparing disease groups in Approach C, which demonstrates another potentially confounding effect, albeit weaker than those already discussed.
Methodologically, our NMR method was limited by a relatively high detection limit, resulting in 33 identified metabolites, of which we excluded the ketone bodies acetoacetate and 3-hydroxybutyrate due to their dependence on fasting levels as well as acetone, acetate and choline due to their dependence on run order. There were five peaks to which we could not assign any known metabolites, and four metabolites (dimethylamine, dimethylglycine, dimethyl sulfone and formate) were identified only by visual inspection and comparison to databases. Our internal standard, which was used for peak scaling, was recently found to be attenuated by the presence of macromolecules  [54] , possibly resulting in lower peak intensities in samples with high concentrations of macromolecules. However, given that in our comparison of methods in our own paper 19/25 (76%) peaks in the final (LED) signature were also found in the ultrafiltration signature  [55] , we concluded that any bias, e.