dataesr/scientific-paragraphs-categorization huggingface dataset:
Biocuration Overview Biocurators strive to present an accessible, accurate and comprehensive representation of biological knowledge  (5) (6) (7). Biocuration is the process of selecting and integrating biological knowledge, data and metadata within a structured database so that it can be accessible, understandable and reusable by the research community. Data and metadata are taken from peer-reviewed publications and other sources and integrated with other data to deliver a value-added product to the public for further research. Biocuration is a multidisciplinary effort that involves subject area experts, software developers, bioinformaticians and researchers. The curation process usually includes a mixture of manual, semi-automated and fully automated workflows. Manual biocuration is the process of an expert reading one or several related publications, assessing and/or validating the quality of the data and entering data manually into a database using curation tools, or by providing spreadsheets to the database manager.
It also encompasses the curation of facts or knowledge, in addition to raw data; for example, the role a gene plays in a particular pathway. These data include information on genes, proteins, DNA or RNA sequences, pathways, mutant and nonmutant phenotypes, mutant interactions, qualitative and quantitative traits, genetic variation, diversity and population data, genetic stocks, genetic maps, chromosomal information, genetic markers and any other information from the publication that the curator deems valuable to the database consumers. Manual curation includes determining and attaching appropriate ontology and metadata annotations to data. This sometimes requires interaction with authors to ensure data is represented correctly and completely, and indeed to ask where the data resides if they are not linked to a publication.
In well-funded large GGB databases, manually curated data may be reviewed by one, two or even three additional curators. Manual biocuration is perhaps the best way to curate data, but no GGB database has enough resources to curate all data manually. Moreover, the number of papers produced by each research community continues to grow rapidly. Thus, semi-automated and fully automated workflows are also used by most databases. For example, a species-specific database may want to retrieve all Gene Ontology (GO) annotations for genes and proteins for their species from a multi-species database like UniProt (http://www. uniprot. org). In this case, a script might be written and used to retrieve that data 'en masse'. Prediction of gene homologs, orthologs and function can also be automated. Some of these standard automated processes require intervention at defined points from expert scientist to choose appropriate references, cut off values, perform verifications and do quality checks.
All biocuration aims to add value to data. Harvesting biological data from published literature, linking it to existing data and adding it to a database enables researchers to access the integrated data and use it to advance scientific knowledge. The manual biocuration of genes, proteins and pathways in one or more species often leads to the development of algorithms and software tools that have wider applications and contribute to automated curation processes. For example, The Arabidopsis Information Resource (TAIR, https://www. arabidopsis. org) has been manually adding GO annotations to thousands of Arabidopsis genes from the literature since 1999. This manual GO annotation is now the gold standard reference set for all other plant GO annotations and is used for inferring gene function of related sequences in all other plant species (8-10).
Another example is the manually curated metabolic pathways in Ecocyc, MetaCyc and PlantCyc, which have been used to predict genome-scale metabolic networks for several species based on gene sequence similarity  (11, 12). The recently developed Plant Reactome database has further streamlined the process of orthology-based projections of plant pathways by creating simultaneous projections for 74 species. These projections are routinely updated along with the curated pathways from the Reactome reference species Oryza sativa  (13). Without manual biocuration of experimental data from Arabidopsis, rice and other model organisms, the plant community would not have the powerful gene function prediction workflows we have today, nor would the development of the wide array of existing genomic resources and automated protocols have been possible.
Well-curated GGB databases play an important role in the data lifecycle by facilitating dissemination and reuse. GGB databases can increase researchers' efficiency, increase the return on research funding investment by maximizing reuse and provide use metrics for those who desire to quantify research impact. We anticipate that the demand for biocurators will increase as the tsunami of 'big data' continues. Despite the fact that the actual cost of data curation is estimated to be less than 0. 1% of the cost of the research that generated primary data  (14) , data curation remains underfunded  (15, 16).
Biocuration is not keeping pace with research and the generation of data. Databases are focused on serving the varied needs of their stakeholders. Because of this, different GGB databases may curate different data types or curate similar data types to varying depths, and are likely to be duplicating efforts to streamline curation. In addition, limited resources for most GGB databases often prevent timely curation of the rapidly growing data in publications. Researchers use many different databases in their work. We recognize there is a clear need for databases that are tailored for the communities that they serve. For example, SoyBase (https://www. soybase. org) (17)  primarily focuses on resources and tools for soybean breeding, while TAIR  (18)  focuses on gene function for the Arabidopsis research community.
However, as more organism genomes are sequenced, comparisons between and among species become more biologically informative. Multi-organism databases like Gramene (www. gramene. org) and Phytozome (https:// phytozome. jgi. doe. gov), Planteome (www. planteome. org) and European Bioinformatics Institute (EBI)'s gene Expression Atlas  (19)  etc. provide resources for comparative genomics studies and link data types from various public databases including species-specific databases. This means that researchers now increasingly have access to several different databases and benefit from their cross-referencing to compare data across species. Use of common data standards, vocabularies, ontologies and tools will not only make curation and data sharing more effective, but will also make databases easier to use.
Recommendations Developing AgBioData standards of curation. One reason for establishing the AgBioData group was to provide a forum to share curation methods from different GGB databases in order to improve and streamline the process. Although the demand for data curation accelerates while resources for biocuration remain limited, by working together, we reduce redundancy and continually improve our curation methods. Each GGB database is responsible for selecting data and providing consistent curation based on the needs of its users. However, the use of curation standards will increase consistency across databases and promote efficiency and interoperability. To move toward more standard biocuration methods, we recommend the following practices: 1. Continue regular communication between biocurators of different databases.
Adopt existing or contribute to developing new Minimum Information for Biological and Biomedical Investigations recommendations for each curated data type. As minimal data and metadata recommendations are established, AgBioData will provide this information to researchers, funders and publishers through publications and seminars, and will strongly encourage their consistent use. These recommendations will also be provided via appropriate online resources such as AgBioData. org and FAIRsharing. org. 3. Share data curation templates. Many biocurators develop and use template spreadsheets for data collec-tion from literature. At MaizeGDB  (22) , for example, templates have been used to ensure consistency between curators and to ease loading of curated data. Database biocurators who have expertise in curation of certain data types are encouraged to share their data templates and curation guidelines through AgBioData.
Challenges and opportunities To identify current status and challenges in ontology use, an online survey was offered to AgBioData members. The survey results for ontology use in databases for each data type are provided in Table  1  and a summary of other survey questions such as barriers to using ontologies are provided in the supplementary material 1. In addition, the ways ontologies are used in data descriptions in some GGB databases are described in supplementary material 2. To facilitate the adoption of ontologies by GGB databases, we describe the challenges identified by the survey along with some opportunities to meet these challenges, including a review of currently available ontologies for agriculture, ontology libraries and registries and tools for working with ontologies. (34).
Lack of funding and resources to train and pay biocurators. While most databases recognize the importance of using ontologies for efficient data integration and retrieval, GGB databases typically lack sufficient funding to train and pay additional biocurators. The curation work could be somewhat eased by tools for the curation and validation of annotations and by standardized data formats for ontology annotation data exchange. Significant work has been done in these areas. The curation tools for GO annotation include TAIR's in-house curation tool PubSearch (  35 ) and TAIR's community portal the Online Annotation Submission Tool (TOAST) (18), PomBase's Canto  (36) , the GO consortium's Noctua (http://noctua. berkeleybop. org) and Table Editor (http://bit. ly/table-editor) (Table  3 ). To facilitate sharing annotations among resources, there are some existing and emergent standards for ontology annotation data exchange.
The GO Annotation File (GAF) format is the standard for GO annotation data exchange (http:// geneontology. org/page/go-annotation-file-format-20) and Phenopackets (Phenotype Exchange Format or PFX; https://github. com/phenopackets/phenopacket-format/wiki/ Overview-of-Phenotype-Exchange-Format) is an extensible data model and data exchange format for phenotype data from any species (Table  3 ). More details about these tools are provided in supplementary material 4.
Lack of ontologies that fit the domain or insufficient terms in existing ontologies. Some databases develop in-house ontologies since the existing ontologies do not meet their needs. When using in-house ontologies, it is necessary to map their terms to reference ontologies to facilitate ontology development and/or data transfer among other databases. In addition, it is often necessary to use species-specific ontologies. For example, analogous organs across plant species often do not have the same name. To ease this difficulty, the CO and Planteome projects work together to link terms in plant species-specific ontologies to more general terms in references ontologies like GO and PO. In case of incomplete ontologies, there is a need for a tool or system where researchers and biocurators can add terms, which are timely reviewed for inclusion in the ontologies.
Make an effort to create a network of ontologies as a community. In order to provide unified access to different types of agricultural data and enable large-scale analysis, it is crucial to have a network of domain ontologies. Each ontology focuses on its own domain, but often several ontologies are needed to fully describe the data. The OBO Foundry initiative (29) provides a variety of tools including an ontology import feature. As an example of the creation of an ontology network, TO defines phenotypic traits in plants as Entity-Quality (EQ) statements. The quality (Q) terms come from the PATO whereas the entity (E) terms come from PO, GO or ChEBI, depending on the entity. These efforts can reduce the curation time in individual databases since once the data is curated with one ontology such as TO, it can be further associated with other component ontologies.
Facilitate ontology annotation by data generators. We encourage the use of ontologies by implementing rules and procedures where available/applicable, and improving ontologies by enlisting community helps in adding new terms, correcting existing terms as needed and in general, modifying ontologies to be broadly adaptive. A good example of this process occurs in AnimalQTLdb, where ontologies were developed in parallel with improvements to AnimalQTLdb  (40). One way to encourage researchers to use ontologies is to provide a system that requires collection of accepted ontology terms as part of the data and/or publication submission process. Some databases, such as TAIR, Tree-Genes (  41 ) and GDR work with journals to require that authors submit their data to the appropriate GGB database prior to manuscript submission  (20).
There are multiple approaches to this. GDR has downloadable data templates that researchers fill in and submit. TreeGenes has a web form for submitting association genetics and population genomics studies. TreeGene's web form is currently being converted to a Tripal module  (42) , with the aim of fully adopting Minimum Information About a Plant Phenotyping Experiment  (43)  to include the full spectrum of data generation, including design, sampling, sequencing and informatic analysis. The TreeGene system simplifies the submission process through the use of guided prompts to query researcher for the location of the experiment (i. e. latitude, longitude, altitude), the type of experiment (e. g. common garden, reciprocal transplant) and environmental conditions (e. g. average temperature), and to determine which ontologies are necessary.
The Dublin Core Metadata Initiative  (44, 45)  is an example of a core standard. For description of data sets, a more comprehensive standard would be https://www. w3. org/TR/hclsdataset/. For scientific research data, a 'core standard' will not be adequate to describe how the data was generated and analysed. Extensions to a 'core standard' are by definition, not standardized and so, extended fields likely cannot be compared. A 'comprehensive standard', on the other hand, may provide sufficient descriptive fields to enable reuse of research data, but its complexity may create a significant barrier to adoption. Another dimension of a metadata standard is the use of CVs. To compare metadata for multiple data sets, there must be a means of directly comparing the contents of each field. CVs, especially in the form of a hierarchical ontology that contains information about the relatedness of values, are essential to metadata.
FAIRsharing. (formerly BIOsharing; https://fairsharing. org) (46-48) is a UK-based clearinghouse of standards, policies and databases in the life sciences, including the biology and biomedical disciplines. Information about standards, policies and databases is curated by both FAIRsharing staff and community members. The data can be searched by keyword, browsed by type or discovered by way of collections of related resources. An example of a collection is all databases and metadata standards associated with metabolic data. The AgBioData group has started collaborating with the FAIRsharing group, as described below, and is represented at FAIRsharing as an organizational collection (https://fairsharing. org/collection/AgBioData). Member GGB databases are encouraged to ensure that their FAIRsharing records are complete and up to date.
Academic libraries. A resource that is close at hand for many researchers and repositories is an academic library. Library science provided one of the earliest metadata standards for digital objects in 1995: the Dublin Core (http://dublincore. org), which remains a foundational recommendation for all types of digital objects. Most academic libraries have one or more data management librarians. To help researchers comply with funding agency policies for data management, data librarians had developed a number of resources: INSDC. is a consortium of sequence databases that includes GenBank from National Center for Biotechology Information (NCBI), ENA and DNA Data Bank of Japan. This consortium participated in setting policies for collection of standard metadata and providing online forms and metadata templates for submission of metadata with genomic data sets.
MIQAS. (Minimum Information for QTLs and Association Studies) is a collaborative effort between Iowa State University and the Roslin Institute of the University of Edinburgh to standardize the minimum information required for database entry and subsequently to facilitate meaningful meta-analyses of QTL/association results. To date, the effort has produced the following outcomes: • A website that describes the MIQAS (http://miqas. sourceforge. net), • Demonstrated application of the required minimum information standards for livestock QTL/GWAS data in Animal QTLdb (https://www. animalgenome. org/QTLdb/ doc/minfo; 40), • Some meta-analyses that benefited from the minimum information framework  (52, 53).
Often custom metadata is used, which, although better than no metadata, make it difficult or impossible to search across or compare data sets. Another challenge is how to permanently attach metadata to data sets to make them discoverable. Too often the meaning of files is implicit in a directory structure, e. g. version number, which is typically lost when files are shared. README files can go missing, file names are changed or provenance is mentioned in a long ago deleted e-mail. This implies that not only is data persistence important but its metadata must also be persistent and permanently attached. Examples of persistent identifiers attached to metadata are GenBank's BioProject and BioSample records that describe genomic, genotypic and expression data deposited at GenBank  (55). Also, DOIs (see https://www.
Recommendations Our main concerns for metadata are the following: use of persistent identifiers; encouraging and enforcing the use of standard and effective metadata; proliferation and duplication of standards development, including frequent use of custom metadata that can't be compared outside a specific GGB database or data set; the difficulty of choosing among large numbers of recommendations and creating profiles to meet specific needs; and the difficulty of filling in extensive metadata forms. To address these concerns, we recommend that researchers and GGB database curators: Work with librarians. at local institutions for training and to help develop data management plans that include collection of appropriate metadata. Curate data and metadata. Manual curation is vital for GGB databases.
Create a robust network of groups working on metadata. The Metadata and Ontologies Working Group, a sub-group under the AgBioData consortium, is a good place to start. It will be difficult or impossible to find and collaborate with everyone working on metadata, but a network will help to encourage collaboration and to raise visibility of the metadata issue. This network will help newcomers understand who is doing what. The situation now is bewildering to someone trying to use metadata correctly. FAIRSharing, COPO and CyVerse are expected to be good partners in the task of disentangling overlapping standards, protocols and tools. Improve database compliance. Each GGB database should require metadata for data sets they host. They should provide metadata in both human readable formats and machine readable.
Metadata should be included with any web service API provided by a database. GGB databases need to help their users find and use appropriate metadata e. g. by embedding the BioSharing search widget in web pages. Adopt a profile approach. Take what you need from existing standards and link to well-established repositories (e. g. BioSamples for sequencing data, genotypes, gene expression etc. ) Two groups that are likely partners in this are FAIRsharing and CyVerse.
Encourage and enforce use of persistent identifiers for data sets. Journals could require established Persistent Uniform Resource Locator (e. g. DOIs, ARKs, accessions from permanent repositories). Also archive historical identifiers (e. g. as synonyms) and link to persistent identifiers. Improve researcher compliance. Create incentives for researchers. Work with funding agencies, journals, databases and general repositories to enforce the use of metadata standards. Work with data librarians to develop and present workshops and tutorials to train scientists via webinars.
Overview Tools that allow biologists to store, distribute and visualize biological data are essential for collaborative and effective science. This need was identified early on with the advent of whole-genome sequencing. Several tools and database schemas were developed to meet the needs of GGB databases, including AceDB in 1989 for the Caenorhabditis elegans genome (58), which was adopted by and modified for other projects; NCBI Entrez (59) released as an online resource in 1993; Ensembl  (60)  in 1999; the UCSC genome browser (61) in 2000; Gadfly for Drosophila melanogaster  (62)  in 2003; AtDB/Illustra in 2003, followed by TAIR  (63)  for Arabidopsis thaliana; and the Genomics Unified Schema, released in 2005 for Plasmodium falciparum  (64). For germplasm, the Germplasm Resources Information Network (GRIN)-Global platform was initiated in 2009  (65) , and has since been adopted internationally for other genebanks (https://www.
grin-global. org/). With low-cost next-generation sequencing technologies, generating genome assemblies is now possible for organisms with smaller research communities and fewer resources. In addition, technologies enabling highthroughput phenotyping can generate large data sets that would benefit from being linked to genomic and genetic data for the same individuals and species. In light of these technological advances, genome-centric databases tailored toward smaller communities, and across multiple taxa, have proliferated  (66) , and several models for storing and accessing data have become adopted. Here, we focus on platforms for community databases centered around GGB information. We define a 'platform' as the database schema and software that serves biological information to researchers.
Review of available platforms. There are multiple platforms currently available for building GGB databases. Here we review several platforms: three that are well established and three newer platforms that show promise. The criteria for selecting these platforms are the following: 1. Open-source codebase and schema; 2. Useful documentation available for installation, management and APIs (if applicable); 3. Adoption of the platform by more than one group, demonstrating that other groups can successfully set up the platform; 4. User support available (e. g. via GitHub issue tracker); 5. The platform enables public data access (e. g. not just a local installation). The three 'well-established' platforms met all criteria; the three 'promising' platforms met four out of five criteria. This list is not meant to be comprehensive.