dataesr/scientific-paragraphs-categorization huggingface dataset:
Pilot Adopters and Deployments Ultimately, the proof of any technology is whether it is adopted and used. In this section we take a closer look at eight specific, operational deployments and the variety of design decisions taken. These pilots are (1) the Center of Biomedical Informatics (CBMI) implementing the solution in an RDBMS setting with i2b2 as a repository for medical data; (2) the Virtual Atomic and Molecular Data Centre (VAMDC), which is a network of distributed repositories operating via an XML-based exchange protocol; (3) the Climate Change Centre Austria (CCCA) operating a repository of NetCDF files queried via a Thredds server; (4) the Forest Ecosystem Monitoring Cooperative (FEMC, formerly the Vermont Monitoring Cooperative, VMC) processing heterogeneous data ranging from databases to images; (5) the Earth Observation Data Center (EODC) running a massive infrastructure serving Sentinel satellite images;  (6)  the Deep Carbon Observatory (DCO) providing centrally managed object identification and access services for a global community;  (7)  the xData platform operated by NICT for real time event monitoring and predictions; and (8) Ocean Networks Canada providing a comprehensive solution across the wide range of data types from more than 400 different types of instruments, including real-time streaming data as well as from autonomous platforms with specific transmission modes.
Center for Biomedical Informatics (CBMI) The Center for Biomedical Informatics (CBMI) at the Washington University in St. Louis (WU, now known as the Informatics Institute) had an i2b2-based  13  repository of biomedical data containing electronic health record data and other biomedical data. This means that when a patient comes into hospital, data on normal care, diagnosis, procedures, medications or any sorts of allergies are recorded for operational purposes. This data is replicated into the biomedical repository to be used for research  [5]. As part of a learning health system, this system supported a number of research data resources. The heart of the system is the Research Data Core (RDC), which is a confederation of data resources (see Fig. 5 ) including i2b2. It comprised multiple database instances for collecting and storing data as well as a variety of web applications that interact with one or more of these databases.
Collectively, these databases housed over six million patient records comprised of over 48 million visits, 114 million laboratory results, and 122 million text documents with daily updates. The implemented recommendations included R1 -Data Versioning, R2 -Timestamping, R3 -Query Store Facilities, R7 -Query Timestamping, R8 -Query PID, R9 -Store Query and R10 -Automated Citation Texts, and have been described in greater detail in a prior publication. [16]. Through a gap analysis and further   [40]  investigation, it became clear that it was necessary to implement the enhancements at the source repository, which was built using PostgreSQL database management system -essentially a replicate of the clinical data. To employ the changes the approach took the work entirely to the DB level: In order to implement Data Versioning (R1) and Timestamping (R2) each live data table in need of versioning was given an additional column holding a timestamp.
Virtual Atomic and Molecular Data Centre (VAMDC) The Virtual Atomic and Molecular Data Centre (VAMDC) is a political and technical framework for operating and sustaining a worldwide digital research infrastructure built over two European projects  [9, 2]. The e-infrastructure federates about 30 heterogeneous atomic and molecular databases that are used for the interpretation of astronomical spectra and for the modeling in media of many fields of astrophysics. Other application fields, cf. Figure  6a , include atmospheric physics, plasmas, fusion, radiation damage. VAMDC offers a common entry point to all federated databases through the VAMDC portal  14  , providing a set of tools to retrieve and handle the data  [23] ). Each node may use different technologies and tools for storing data and is responsible for its data.
New data may be added or new versions of existing data may be provided. An ad hoc generic wrapping software, called the node-software transforms an autonomous database into a VAMDC federated database, called a data-node. Each data-node accepts queries submitted in a standard grammar (VAMDC SQL Subset) Figure  7 : Functional overview of the query store at the VAMDC  [40]. and, by implementing an interoperable data access protocol, provides output formatted into a standard XML file (VAMDC XML Schema for Atomic Molecular and Solid Data, VAMDC-XSAMS), shown in Figure  6b. The VAMDC data-citation implementation is based over the Node Software technology  [39]. Concerning the data versioning and time-stamping (R1 and R2), VAMDC has two different mechanisms: • coarse-grained: a modification of any publicly available data at a given data-node induces an increment in the version of the data-node.
A notification mechanism informs that something has changed on a given data-node, indicating that the result of an identical query may be different from one version to the other. • fine-grained: The information contained in the Version element in the VAMDC-XSAMS standard. [41]  indicates which data have changed between two different data-node versions. For extracting data from VAMDC, users may query directly a given known data-node or use one of the centralized query-clients (e. g. the already mentioned VAMDC portal). In the latter case, the centralized client software asks the registries what are the data-nodes able to answer and dispatches to them the query. Any centralized client acts as a relay. This is completely transparent from the data-node perspective and a data-node acts in the same way regardless the source of the query it is serving; when a data-node receives a query: • it generates a unique query-token (this can be seen as a session token associated to the incoming query); • it answers the query by producing the VAMDC-XSAMS output file, which is returned to the user together with the generated query-token.
The stable sorting (R5) is guaranteed by the node-software. The token is copied both in the header of the answer and in the output file; • it notifies the Query-Store (R3), providing the query-token, the content of the query, the version of the node and the version of the standards used for formatting the output. It is worth noting that this process is not blocking and has no impact on the existing infrastructure whatsoever: the data extraction process is not slowed down. If the Query Store cannot be reached the user will still receive the VAMDC-XSAMS output file. When the Query-Store service (R3) receives a notification from the data node, it stores the received information and reduces the query to a standard form, using the VAMDC SQL-comparator library 15  (R4), and it checks if a semantically identical query has already been submitted to the same data-node, having the same node version and working with the same version of the standards.
• If there is no such a query, the Query-Store service attributes a unique UUID and a timestamp to the new query (R8 and R7), downloads the data, i. e. the VAMDC-XSAMS output file from the data-node and processes this file in order to extract the bibliographic information (each VAMDC-XSAMS file produced by the VAMDC infrastructure includes the references to the articles used for compiling the data) as well as metadata. The relevant metadata are stored and associated with the generated UUID. These metadata are kept permanently (R9): the Query-Store service permanently keeps the mapping between the UUID and the set of query-tokens assigned to a given query. The functioning of the Query-Store is asynchronous. This was a mandatory constraint in order to avoid slowing down the VAMDC-infrastructure with a central bottleneck service.
The unique identifier assigned to each query is resolvable (R11), and is both human and machine actionable (R12). The associated landing page provides the metadata associated with the query, as well as the access to the queried data and a BibTex citation snippet (R10). The VAMDC Query Store is interconnected to Zenodo: from the landing page displaying the information about a given query, the user may with a simple "click" trigger the replication of the data on Zenodo, together with all the metadata and bibliographic references. This procedure also generates a DOI as alternative PID for the current query. The interconnection with Zenodo 16  provides the Query Store with Scholix 17  functionalities and with lifetime access to the query-generated data. This query store's benefits include a) its transparent usage for users, b) the live monitoring of queries and users, so that data providers can measure their impact, c) easy and automatized data citing and d) minimal impact on existing infrastructures.
Climate Change Centre Austria (CCCA) The Climate Change Centre Austria (CCCA) is a research network promoting climate research and climate impact research. The CCCA-Data Centre, as one of the three CCCA departments, operates a research data infrastructure for Austria with storage capacity of more than 700 TB embedded in a highly available Linux Server Cluster and linked to the high-performance computing facilities of the Vienna Scientific Cluster and the Central Institute for Meteorology and Geodynamics (ZAMG) as the national weather service. The service portfolio includes a central access point for storing and distributing scientific data and information in an open and interoperable manner. Starting with the initial tasks of setting up a data management framework for heterogeneous climate information, the focus changed in 2016 to highly resolved regional climate scenarios.
The interval between updates or new versions usually depends on the time frame of the funding schemes and their research projects and happens approximately once per year. The main motivation for setting up a web-based tool for dynamic data citation and its fragments was to have a technical solution to align a persistent identifier with an automatically generated citation text. The core component for the set up within the CCCA environment depicted in Figure 8  [34]  is the Handle. NET®Registry Server for PID assignment. For processing and creating data fragments, the Unidata Thredds Data Server (TDS) in combination with NCDF Subset Services (NCSS) is embedded. NCSS provides a catalog of parameters that allows the description of data fragments while retaining the original characteristics of the data.
Forest Ecosystem Monitoring Cooperative (FEMC, formerly VMC) The Forest Ecosystem Monitoring Cooperative (FEMC, formerly the Vermont Monitoring Cooperative, VMC) is a collaborative network that monitors forest ecosystems. The FEMC provides a data archive and an access and integration portal for the network to save data and to make them more available for assessment and research. Understanding complex ecosystem processes requires cross-disciplinary work. Thus, the data comes from a many different fields in natural resources science and from diverse contributors, from citizen scientists to monitoring professionals to researchers. Much of the data in the system are highly dynamic as data are added or updated frequently, necessitating versioning in order to cite the data. These data include observations of tree canopy condition, soil chemistry, high elevation bird counts, and photomonitoring of alpine vegetation quadrants.
When possible, data are stored as tables in a relational database system that includes a metadata documentation workflow. Data that cannot be stored as a database table -such as raster, vector, and image formats -are stored in a file system. Users manage and access data through a web interface. Because the FEMC archive supports both monitoring and research use cases, the ability to cite an evolving data set in a way that allows others to access that the exact state of the data used in a particular analysis is critical. FEMC sought to implement R1 -Data Versioning, R2 -Timestamping, R3 -Query Store Facilities, and R7 -Query Timestamping. The workflow that emerged to implement the recommendations covers three different steps: data addition and editing, sub-setting, and recovery. The data editing allows for provenance tracking so that a data set can be modified without affecting the previous iterations of the data set.
Users add data to the system, at which point they can commit this to a version, preventing additional changes to the stored content. Modifications or additions to the data set are tracked as subsequent versions of the same data resource, preserving the evolution of the data set over time. When tabular data are updated to a new version, the database table storing the data is updated to the new state, and the database operations required to roll back the database table to the previous state are recorded. The system creates a result hash and a query hash to accompany the version identifying information as a way to check the validity of any rollback operations. For file-based data, the file is given a unique name, registered in the version table, and stored in the file system, but the result hash is not created.
FEMC sought a parsimonious solution that did not require storing copies of data in every situation, leading to a lot of work to differentiate between additions and replacements. The implementation of the query store and the initial steps towards verification of query and result sets was relatively straightforward, requiring less than a person month of time. However, this upgrade solved several other issues in the FEMC archive, and thus taking these steps to lay the groundwork for the recommendations was welcome. The biggest improvement aside from addition of DDC capacity was the structure built to version datasets themselves. Previously, the same data were being uploaded as entirely new data sets every time a change was made, and users then couldn't figure out which to use, or the previous data was being overwritten to provide the one authoritative data set, breaking previous uses of the data.
FEMC's long-term monitoring reports  19  were produced as a snapshot of the previous year's data in over a dozen key metrics. As monitoring datasets evolved, our site held only one representation and thus FEMC was invalidating links in its own publications by subsequently updating data. It also improved the culture of data management within the organization for FEMC's own monitoring work, by providing a clear point where FEMC staff certify the data produced as final and ready for distribution, such as our annual forest health monitoring work  20. Researchers have appreciated the certainty that comes from versioning, and the dynamic data citation capabilities are now fully integrated into the data processing workflow in our forest indicators dashboard  21. The Earth Observation Data Centre for Water Resources Monitoring (EODC) is a processing and data backend founded in 2014 and located in Vienna, Austria.
It operates a multi-petabyte, scaled, storage infrastructure connected to the Vienna Scientific Cluster (VSC) high-performance computing (HPC) system. It obtains Sentinel 1-3 data from the European Space Agency (ESA) Program Copernicus. New data are added in up to daily increments, depending on the satellite and sensor type. ESA releases data updates and corrections in cases when one of the instruments used for the observation was wrongly calibrated or broken. These are rare and never happened in the history of EODC so far. To increase reproducibility of studies and support precise data citation, the existing systems were modified to support precise data identification. The implemented recommendations include R3 -Query Store Facilities, R4-Query Uniqueness, R6-Result Set Verification, R7-Query Timestamping and R8-Query PID, with versioning and timestamping (R1 and R2) already in place via the standard storage infrastructure.
Researchers do not run their experiments locally but on the environment of the central server. Therefore, a definition of the processing steps and the input data is transmitted e. g. following the openEO standard  [27]. The backend creates a new job following the definition and waits for the researcher to start it. After doing so, the backend starts the processing, providing the researcher with status information on request. When the backend finishes the processing, the researcher can request the result by following the provided download link. After adding our data citation extension to the backend, the workflow was unchanged, so researchers do not have to change the way to work with the backend. EODC is a file-based earth observation backend, which uses a PostgreSQL (incl. PostGIS extension) metadatabase with an Open Geospatial Consortium (OGC) compliant Catalogue Service for the Web (CSW) interface for querying.
It represents a central service for data-driven applications of EODC using queries via XML requests and responses on a publicly available endpoint  22. The unique path to a file is the identifier and used for the versioning, since every data update results in a new path. The creation timestamp is persisted in the metadatabase, which we use to query for data versions that were available at a particular time (see red marker in CSW Query excerpt provided in figure  11 ). The query store is implemented as an additional table in the metadatabase for the job executions. The unique query is defined as an alphabetically sorted JSON object of the filter arguments since the order of the filters makes no difference in the outcome. The filter arguments consist of the satellite identifier, the spatial extent, the temporal extent, and the spectral bands of the satellite.
The result of the query is a list of files with a fixed order defined by the CSW standard  [15]. For fast comparisons, the hash of the resulting file list and the hash of the unique query are stored in the query table. During every job execution the query PID of the input data is added to the job meta-data, either by generating a new query PID or by using an existing query PID if the same query was executed before. Figure  12  provides an overview The effort for implementing the solution was roughly one person month. The most challenging part was the integration with the standards underlying the system of EODC (OGC, CSW, and PostGIS) and to set up a test instance of the actual server. The implementation was relatively straight-forward, since the versioning and timestamping was already in place for the GeoGIS database.
Deep Carbon Observatory The Deep Carbon Observatory (DCO) is a global community of multi-disciplinary researchers unlocking the inner secrets of Earth through investigations into life, energy, and the fundamentally unique chemistry of carbon. It started as a 10-year initiative that produced significant data and scientific results. 23 DCO organized a Data Portal that provides a centrally managed digital object identification, object registration, and metadata management service that provides discovery and access to diverse data for the DCO community. The DCO Data Science Team at the Tetherless World Constellation of Rensselaer Polytechnic Institute maintains the DCO Data Portal outlined in Figure  13. The Portal makes extensive use of Persistent Identifiers, most notably something called the DCO-ID.
The DCO-ID is a Handle and is similar to the Digital Object Identifier (DOI) for publications, but it extends the scope to many more types of objects, including publications, people, organizations, instruments, data sets, sample collections, keywords, conferences, etc. Each DCO-ID can redirect to the Web profile (often a landing page) of an object where detailed metadata can be found. In the DCO Data Portal, each object is the instance of a class. The metadata items describing an instance are properties. All those classes and properties are organized by the DCO ontology. [20, 29]. In implementing the WGDC Recommendation, the team essentially walked through the 14 recommendations, assessed where they were in terms of compliance, and then made adjustments accordingly. Most of the work was conducted by a summer undergraduate student hired as part of an RDA/US adoption grant.
xData Platform at NICT The Big Data Integration Research Center at the National Institute of Information and Communications Technology in Tokyo (NICT) develops a data analysis platform, called xData Platform, on NICT Integrated Testbed. It aims at collecting heterogeneous sensing data from various data sources, then discovering and predicting associations of complex events in the real world for providing actionable information  [38]. For example, it enables predictive modeling of traffic obstructions caused by extraordinary weather based on association mining between weather observation data and traffic monitoring data for route navigation applications. Multiple domains of data sets are being collected by platform users such as weather observation data (precipitation radar, meteorological stations), atmospheric observation data (air pollution observations, personal environment sensors), traffic monitoring data (congestion, prove car) and lifelog data (fitness sensors, camera sensors).
The xData Platform consists of a database server, called Event Data Warehouse, and APIs for collecting, associating, predicting and distributing data sets. The Event Data Warehouse transforms the data sets into a common format, called event data, for storing heterogeneous sensing data in an interoperable manner. Through the APIs, a transaction data table is created by joining multiple event data onto spatial and temporal attributes. Cleaning and tailoring the transaction data, predictive modeling of associative events is processed based on data mining and machine learning methods like frequent itemset discovery and deep neural networks. The prediction results are then distributed in application-friendly formats like JSON. For the purpose of tracing and verifying the cross-data analysis process, the xData Platform provides a data provenance function.
As shown in Fig. 15 , the data provenance function captures a workflow showing how data sets are selected, processed and generated by the APIs during execution of an analysis process. A programming library is provided for capturing the provenance information (e. g. , provenance. process(API) ) Dynamic data citation is a key for realizing the data provenance function for sensing data. Adopting the R3 (Query Store), a dynamic data citation is implemented based on a "view" of data table in Event Data Warehouse, which is a database object containing a query to a database table for selecting a target data set dynamically. Concerning the R1 (Data Versioning) and R2 (Timestamping), dynamic data citation is represented by combination of a table name and a timestamp of view creation. Fig. 16  shows an example of dynamic data citation, where two different views are created from a growing archive table of rainfall sensing data "xrain contour" for citing the data set available at two different execution times of a workflow.
Ocean Network Canada Ocean Networks Canada (ONC) operates observatories and platforms in coastal, deep-ocean and polar environments. The majority of data streams are real-time from cabled observatories such as the NEPTUNE Figure  17 : The ONC Oceans 2. 0 system (in blue), and third party sources and applications (in orange). Dotted lines indicate aspects that were added, while all ONC components were modified. Modifications included an extended data model, additional web services, integration of third party APIs and data citation features. observatory in the North East Pacific, but there are also autonomous and mobile platforms with other data transmission modes. In addition to serving these data management needs, ONC fulfills a repository role for partners spanning government, non-profit, industry, and First Nations.