dataesr/scientific-paragraphs-categorization huggingface dataset:
Breedbase has worked to build in native support for these applications and integrate them into best practices workflows. Since internet access is not available at all field sites, the functionality has been developed to allow configuration of these applications prior to field data collection. Field layouts, plant accessions, and traits to be measured can be loaded onto mobile devices through special interfaces in Breedbase. Following collection, data is imported back into Breedbase. Because all the trial information in the collection device was initially downloaded from Breedbase, required identifiers can easily be matched with the existing data in the database. This process is called "roundtripping", and is a crucially important concept for high quality data management.
List management Breeding activities often require the maintenance of lists of various types -for example, a list of accessions to plant, traits to measure, or trials to evaluate -and, consistent with digital ecosystem principles, these lists should be managed entirely through the database. Accordingly, Breedbase implements comprehensive list management functions. By default, lists are associated with the user that creates the list. The main list interface can be reached by clicking on the Lists link on the top right of the toolbar, which appears when logged in. A dialog appears that allows users to view, create and edit new lists. Each list has a data type from an internal ontology called 'list_type', which includes terms for 'accessions', 'trials', 'traits', 'years', etc. Lists are collections of text elements that correspond to names of database objects.
Germplasm management Germplasm is the foundation of a breeding program and plays a similarly important role in a database such as Breedbase. In plant breeding programs, tracking and characterization of germplasm is a major challenge. Germplasm in this context includes accessions, stocks, varieties, or, in clonal crops, clones. Breedbase commonly uses the term "accession". Breedbase is pre-populated with the complete plant section of the NCBI taxonomy database, defining all known species with their associated genus, abbreviation, common name, and GenBank taxon identifier. Researchers using Breedbase can usually find their crops of interest within the 100,000+ organisms available. Accessions are always created in association with one of these organisms. Some instances of Breedbase, such as Cassavabase and Sweetpotatobase, are designed to only contain germplasm of their respective species; however, it is possible for a single instance of Breedbase to be used for a variety of crop species.
Combining many crop species into a single instance can complicate the search interfaces and lead to bloated databases; however, aggregating all data allows for more consistent and queryable data. Alternatively, separating instances can lead to potentially duplicated and inconsistent data, but can be beneficial for fostering communities. In Breedbase, there are two distinct concepts that describe accessions: (1) an accession that can be ordered from a seed bank, which may have been selfed and could be genetically quite pure, or landraces. These are "long-term use" accessions (ie: historical germplasm, parental inbred lines) , which may be actively maintained and can be obtained easily; whereas (2), are "short-term use" accessions (ie: intermediate generations) that are produced in a breeding program and may go through a few rounds of selection, but most of which will be discarded in the process.
These accessions may also not be genetically pure, as they may result from crosses between relatively distant parents. To create an accession in Breedbase, only a unique name and the organism species name are required. As with all objects stored in a relational database, Postgres will create a primary key identifier for each object, using a data structure called a sequence, which is used to link the accession to other objects in the database using a foreign key. This means that even if the accession name is modified, it will still retain all the connections to other objects of the original entry. Germplasm can be further annotated with configurable properties from the Multi-Crop Passport Descriptors (MCPD) standards (Food and Agriculture Organization of the United Nations 2018) and BrAPI standards  (Selby et al.
2019 ); these properties include 'variety', 'donor', 'donor institute', 'donor PUI', 'country of origin', 'institute code', 'institute name', 'notes', 'accession number', and 'PUI'. Germplasm can be added to the database using the interactive list tool (see previous section) or an Excel file upload; the Excel file upload also allows for storing and updating of all attributes listed above. The first step in the initiation of a breeding program is to load relevant accessions into the database. This is critical, as the naming of accessions is often not uniform between breeding programs and the community at large. In some cases, a single name can refer to several different accessions or a single accession may have many different names or synonyms, often the result of historical transcription error or case inconsistency.
Before the first upload it is therefore essential to define a standard unique name and set of possible synonyms for each accession. Though Breedbase allows for synonyms of accession names, they should also be unique. It is best practice to use synonyms only to find accessions and not when performing routine tasks with the database during the breeding process. Whenever new accession names are encountered, Breedbase provides a workflow to compare new names to all existing accessions in the database. In this workflow, a user can consolidate synonyms, for instance to add 'Tx 303' as a synonym of 'TX303'. After initial accession upload, it is often necessary to add more accessions, increasing the chance of generating duplicated accessions in the database, or other upload issues. As is the case with synonyms, many of these problems result from poorly defined accession identifiers with capitalization inconsistencies and special characters such as slashes, dots, dashes, underlines, and spaces.
Within Breedbase, a field trial links phenotypic observations to the experimental layout under a specific statistical design. Row crops usually use the concept of plot as the minimal entity for data collection, but many specialty crops (i. e. , vegetables) require data collection on a per plant or per tissue basis. Breedbase allows plant-and tissue-level entry creation for each plot in a trial, resulting in database entries and identifiers at each level, which can also be encoded in barcode labels for data collection.
The plot or plant names of the parents are related to the field trial in which they are planted, as is described in the above field trial section. This approach allows detailed tracking of female and male parents used in crossing, but is optional in Breedbase because of the difficulty in recording this information in many cases. Recording information on parental plots is facilitated by mobile data collection platforms. Of note are customized Open Data Kit (ODK) Android applications, such as BTract and the PhenoApps app Intercross. BTract assigns and prints a unique cross barcode label after scanning barcodes to track the precise male and female plots or plants involved in the pollination. Through ODK data synchronization, the cross information can be uploaded into Breedbase. Intercross can be used to scan parental barcodes and associate a unique cross id to the performed cross.
Progeny of the cross can be saved as new germplasm in the database, automatically creating pedigrees for the new germplasm. High-density genotyping data are a complex data type that have become an important resource in modern breeding programs due to the advent of low-cost next-generation sequencing (NGS) and genotyping technologies  (Thomson 2014). Breedbase offers simple laboratory information management functionalities from field tissue sampling to SNP data storage. Functions include tissue samples collection and tracking via plot barcodes and PCR plate formats (ie: 96 or 384 wells), genotyping protocol definition, data storage and subsequent analytics  (Morales, Bauchet, et al. 2020; Tecle et al. 2014).
2011; Semagn et al. 2014; Kilian et al. 2012). The preferred method for uploading high-density genotyping data to Breedbase is through variant call format (VCF) files. VCF provides for compact representation of genotypic scores for large numbers of samples and markers  (Danecek et al. 2011). PostgreSQL non-relational functionalities allow Breedbase to store highdensity genotyping data in JavaScript Object Notation (JSON) structures within the larger relational database schema ("ISO/IEC TR 19075-6:2017" 2018). Breedbase Downloaded from https://academic. oup. com/g3journal/advance-article/doi/10. 1093/g3journal/jkac078/6564228 by guest on 14 April 2022 particularly relies on the binary JSON (JSONb) data type for compressed data storage and faster retrieval  (Morales, Bauchet, et al. 2020). Genotyping data can be queried alongside relationally stored phenotypic and experimental information for analyses, including computation of a genomic relationship matrix (GRM) for user specified germplasm and computation of a genome-wide association study (GWAS) for user specified germplasm and phenotypic traits  (VanRaden 2008).
Authentication and Authorization During breeding processes, a potentially large number of people will need to access the database to download, upload, modify or delete data. This requires a fine-tuned layer of authentication and authorization management in the database. Breedbase requires a user to login for most functionalities (authentication). Every user account is associated with "roles" that determine what the user will be allowed to do in the system (authorization). Currently, there are three major roles: user, submitter and curator. The user role allows read-only access. With the submitter role, a user can upload data, and can modify or delete data that they themselves uploaded. The curator role allows a user to modify any type of data. In addition, every breeding program in the database has a corresponding role that controls authorization over specific breeding program activities, such as creating and uploading trial data.
Cassavabase, the flagship Breedbase database Cassavabase (https://cassavabase. org/) is the breeding database for the NextGen Cassava project (https://nextgencassava. org/). The NextGen Cassava partners, IITA (Ibadan, Nigeria), NRCRI (Umudike, Nigeria), NaCRRI (Namulonge, Uganda), TARI (Ukiriguru, Tanzania), Embrapa (Cruz das Almas, Brazil) and CIAT (Cali, Colombia) use Cassavabase for their breeding programs, starting as early as 2014. To date, Cassavabase has accumulated an immense amount of cassava breeding data (Figure  1c ), consisting of information on more than 500,000 cassava accessions, characterized by over 19 million phenotypic measurements in over 4,000 trials, and nearly 35,000 genotyping experiments. This shows that the Breedbase system can scale to fairly large datasets and large, multi-institute and multi-national programs.
org/, (Li et al. 2018) , Erysimum (https://erysimum. org/, (Züst et al. 2020 )) and milkweed (https://milkweedbase. org/). In addition, a Breedbase instance has been deployed to characterize a tritrophic vector-borne disease system, the citrus greening disease (https://citrusgreening. org/)  (Saha et al. 2017 ). An instance named ImageBreed has been deployed for high-throughput imaging of maize and alfalfa field experiments (https://imagebreed. org/)  (Morales, Kaczmar, et al. 2020). A number of academic labs and breeding companies also use Breedbase for data management within their programs. The Breeding Insight project (https://breedinginsight. org/), which creates breeding databases for USDA breeding programs, has also adopted the Breedbase system as a foundation for their breeding solutions.
Toolbox adaptation of Breedbase As documented in this article, Breedbase provides many features for working breeding programs. The mission of The Triticeae Toolbox (T3) is to provide these features to a diverse audience of small grains breeding programs, by mandate in the United States, and by extension globally. The development of T3 is motivated by the belief that larger datasets provide greater power to identify genetic effects that are relevant to all breeders. Across wheat, oat, and barley, T3 stores 5,600 trials, comprising over 1,800,000 phenotypic data points on over 30,000 lines with genotype data. From there, T3 seeks to provide breeders with results from analyses that tap into these data, in the hope that this will help breeders gain insights from their own data. The primary example we have in this area is a function to show marker trait associations identified among all trials submitted to T3 with adequate marker density, and meta-analyzed to determine robust associations across trials.
The next milestone on the roadmap of this function is to develop marker imputation functionality on T3 that will present genotype trials with uniform high-density marker scores, enabling meta-analysis over more trials. Indeed, marker data are a critical rationale for T3's mission: the database contains data on many lines that now are connected to current populations primarily through the marker alleles segregating. An important advantage of a web-based data management platform is that it links the data to the world of knowledge available on the web. T3 provides that connectivity by providing links to external information on markers, traits, and germplasm. Our primary partners in that regard are GrainGenes, Wheat Expression Browser, and the Wheat KnetMiner  (Hassani-Pak et al. 2021). For example, a marker trait association close to a gene can be used to connect that trait to JBrowse (https://jbrowse.
org/), to gene expression data (expVIP and EMBL-EBI) or to a knowledge network, KnetMiner. Traits in Breedbase are defined using collaborative ontologies crucial to forging these links: the ontologies represent agreements on naming traits and gene functions that enable meaningful bridges across knowledge platforms. The diversity of T3 users means that they will not operate together as an integrated breeding organization. Rather each breeding program submitting data to T3 will want data privacy and ease in determining what data becomes incorporated into the public production database. Currently, all data on the production database is available to anyone. We plan on implementing privacy settings specifying data visibility as public or restricted. Absent this feature, we now work with a few users by providing them with separate instances of T3 that are not publicly visible but can easily transmit datasets to the T3 production database when ready.
The wide range of T3 users also means that we expect them to have varying degrees of familiarity with the Breedbase platform. To allow users to test the addition and modification of datasets without modifying curated data by mistake, T3 has created sandbox instances for each crop. Users can freely upload data to the sandbox, ensure that the uploaded data added to the database is correct, and then easily publish the data to the production instance. A data curator checks the submitted data before adding it to the production database. The Breedbase system was crucial in establishing these features and reduced duplication of effort.
DISCUSSION Breeding is a complex process involving many different types of data, especially considering genome-based breeding methods at the current state of the art. Creating and maintaining breeding databases is therefore generally considered to be timeconsuming and expensive. Many large breeding companies maintain their own databases and software for managing breeding processes and selection, but this is not an option for smaller programs. The lack of bespoke databases is especially true in resource poor areas of the world, where the need for plant improvement is often the greatest. A free, user-driven and open source platform such as Breedbase that integrates a complete digital ecosystem for breeding will help close the gap for these programs as well as many smaller to mid-sized organizations.
Integration in breeding programs Even the best breeding data management tools will fail to deliver if breeding programs do not use them or use them incorrectly. A significant effort is required to integrate a breeding database into the workflow of a breeding organization, as data management is central to the work of modern breeding programs but remains a shortcoming. Breeding activities need to be closely tracked; to ensure complete integration, all materials, operations, and operators need to be systematically recorded and reviewed throughout the process. This is important to enable analyses, improve data quality, and to identify sources of errors in real time and post hoc. It is important for breeding programs to work closely with groups that have significant experience in data management, which can also help the breeding programs to understand their needs, and to train staff better in the use of the database.
In the RTB defined processes for updating and developing the ontologies, which allows new traits and methods to be introduced to breeding programs with relative ease. Future developments Progress in the last few years in digital agriculture has been enormous and will continue to be so in the foreseeable future. New genotyping and phenotyping technologies, such as near-infrared spectroscopy, are constantly being developed or improved. Breeding databases must co-evolve with the technological advances to remain relevant, requiring significant effort in refactoring and implementation. Systems that easily adapt to new technologies will have a distinct advantage; in terms of software development strategies, agile software development will be more efficient than older waterfall type models. Another area of improvement is that of algorithms and other aspects of methodology.
With a strong connection to the R programming language, it is relatively easy to implement new algorithms in Breedbase, as they often require little modification from standalone scripts to work within Breedbase. At its core, Breedbase uses a relational database with integrated JSON data storage, which provides a healthy balance between highly structured, normalized data and flexibility. However, other systems, such as graph databases and highly parallelized solutions like Hadoop, or a combination thereof, are becoming popular and may be integrated into Breedbase in the future. All of the Breedbase code is open source and readily available on the code sharing site GitHub (https://github. com/solgenomics).
Conclusions Breedbase provides a fully open-source, scalable and feature-rich breeding digital ecosystem that has been in use at the RTB crops breeding centers of the CGIAR for Breedbase operates on a relational database using Postgres. Postgres 12. 0 offers "Big data" solutions including parallel query execution and optimized binary javascript object notation data type (JSON) handling. Binary JSON (JSONB) is a simple data structure designed to be storage space and scan-speed efficient. In Breedbase, JSONB is used in various data types including genotypic (marker) information. In addition to the relational database a standard file system space is available for flat files. Finally, other databases can communicate to a Breedbase instance to provide additional back-end for marker data (ie: Genomic Open Source Informatic Initiative (GOBii)) or to exchange germplasm information for example.
management stages necessary to facilitate genomic selection within a breeding program, including:  Manage accessions and pedigrees in the database, with ontology-based descriptions and support for rich metadata including images  Design field layouts and track all field metadata  Load historical data from breeding programs  Collect phenotypic data on tablets in the field and upload the subsequent phenotypes  Manage genotypic data associated with the accessions  Enable genome-based predictive breeding by calculating correlations between phenotypes and genotypes, and predict phenotypes from genotypes [the solGS tool (Tecle et al. 2014), https://cassavabase. org/solgs/search]  Support controlled crossing using customized tracking tools More recently, a number of other use cases were pursued:  Advanced statistical analyses including Principal Component Analysis (PCA), stability analysis (AMMI) (Duarte and Pinto 2002) heritability calculations (Holland, Nyquist, and Cervantes-Martínez 2010), mixed model analysis, and genome-wide association studies (GWAS)  Marker-assisted breeding  Processing and analysis of unoccupied aerial vehicle (UAV) image data  Image analysis Downloaded from https://academic.
unit and integration tests. A ticket meeting is held once a week and all open pull requests and important tickets are discussed. If all the pull requests were merged successfully, and no issues are discovered with tests or other checks, a new release tag is created, the new version is deployed in production, and a new Docker image is released. Since Breedbase is open source, programmers outside of the core development team are able to make contributions to the code base via the same process. The Breedbase project has had 40+ contributors addressing various issues and improvements (https://github. com/solgenomics/sgn/graphs/contributors).
Acknowledgments Downloaded from https://academic. oup. com/g3journal/advance-article/doi/10. 1093/g3journal/jkac078/6564228 by guest on 14 April 2022 Funding: This work was partially supported by the  NEXTGEN   Cassava  project, through a grant to Cornell University by the  Bill & Melinda Gates Foundation  (Grant  INV-007637  http://www. gatesfoundation. org)and the  UK's Foreign, Commonwealth & Development Office (FCDO). Other funding was provided by  BMGF  through the  Africa Yam project ,  Better Breeding Bananas , and the sweetpotato-focused GT4SP and SweetGAINS projects. JLJ,  CLB , and DJW received partial support from the  US Wheat and Barley Scab Initiative  and the  National Research Initiative Competitive Grants   2017-67007-25939  and  2017-67007-25929  from the  National Institute of Food and Agriculture, U.
Availability https://github. com/solgenomics https://hub. docker. com/r/breedbase/breedbase License -MIT License. many years, starting with the NextGen Cassava database, Cassavabase (https://cassavabase. org/). The system has now been adopted by various breeding programs including vegetable and grain crops and maintains an open and collaborative approach to software development, allowing database customization for each research community while sustaining a common framework. Our hope is that Breedbase, and the digital ecosystem that it provides, can contribute, in a small way, to solving the world's big problems with food scarcity and food quality, and thus contribute to improving subsistence farmers' lives around the world. Web Resources https://github. com/solgenomics/ Github repositories for Breedbase code https://hub.
docker. com/r/breedbase/breedbase# Docker image for Breedbase server https://breedbase. org/ Breedbase demo site https://cassavabase. org/ Cassavabase, the flagship Breedbase site https://musabase. org/ Breedbase site for banana breeding https://yambase. org/ Breedbase site for yam breeding https://sweetpotatobase. org/ Breedbase site for sweet potato breeding https://www. youtube. com/channel/UC3jrvvzGKKEHzOriDBgnj0A YouTube channel for Breedbase Data availability statement All code is available from Github (https://github. com/solgenomics) and docker hub (https://hub. docker. com/r/breedbase/breedbase#).