dataesr/scientific-paragraphs-categorization huggingface dataset:
In turn, the DOI can then be inserted back into the SPASE description to cement their linkage. The utilization of DOIs within SPASE descriptions allows for precise and consistent citation of data resources, a clear benefit for the Heliophysics research community. In the case of the ESDC datasets, the ESA approach has been to first register collections of heliophysics data associated with each ''experiment" (an instrument or a set of related instruments) for any ESA heliophysics mission  (Masson et al. , 2021). This provides users with the ability to cite, in a paper, any bunch of datasets produced by one experiment with one DOI, and acknowledge the experiment Principal Investigator. There are on-going efforts to also generate a DOI per dataset. Another level of granularity is that of the ''collection" (rather than a dataset from a single instrument as above).
A collection can be a composite dataset, as described later in section 4. 1. 3. 1, that is a heterogeneous grouping of digital objects united by some purpose, such as in support of a particular investigation reported in a journal article. In astrophysics it is common to collect the pointers to all the observations needed for the article and to bundle them with one DOI. ESA has generated thousands of DOIs related to data bundles associated with each selected observation proposal for observatory-type astrophysics missions (e. g. , XMM-Newton) or data releases for survey-type astrophysics missions like Gaia. In the planetary domain, ESA has assigned a DOI to all planetary datasets which are, by definition, bundles in this field (see  Masson et al. , 2021  for more details). Eventually, ESA envisages allowing users to mint DOIs on the fly in its archives related to datasets, versions, and time periods used in a particular research study in order to include that such DOI in their paper.
This will enable individual scientists to directly link their paper to the datasets used, enhancing the reproducibility of their results. There might also be a DOI associated with data output from software, such as a simulation model. For these types of objects both the software and the output will have DOIs which will aid in the reproducibility and validation of results. This style of DOI usage is becoming more common in Heliophysics as Open Science (see section 5. 1) gets more widely adopted. One could also imagine a study partially based on heliophysics datasets/software/simulation described in SPASE combined with datasets (e. g. , planetary, astronomy) not described in SPASE. In this context, one service worth highlighting is Zenodo (https://zenodo. org), which allows a user to submit any digital object and generate a DOI.
It is indeed one of the general repositories (https://doi. org/10. 5281/zenodo. 3946720) listed in the Data and Software sharing guidance for authors submitting to AGU journals for instance (https://doi. org/10. 5281/zenodo. 5124741). However, it should be noted that this AGU sharing guidance recommends its authors to first look for a repository that specializes in the data of the authors' scientific domain (see https://data. agu. org/resources/useful-domain-repositories), like the SPDF and SDAC for heliophysics data, as this will maximize the probability that the deposited data will be FAIR-compliant.
2  in  Roberts et al. (2018)  for the full list of header information). AccessInformation provides information on how and from where to access the resource electronically, such as through a web interface or web services. This is also where data format information is provided so that users would be able to know how or what tools would be needed to access the content of the data. MeasurementType indicates the nature of the type of observational measurement, such as ion composition, field, or wave, etc. The information required to describe a digital resource as a whole, such as an entire dataset, may be sufficient for searching and locating the resource; but the globallevel description alone is insufficient knowledge for accessing and understanding the contents of the dataset. As discussed in section 2.
2. 4 above, descriptions of data format (2. 2. 4. 1) and parameters (2. 2. 4. 3) effectively provide the balance of the necessary information on accessing and understanding the data contents. Figs. 3  and 8  show that both AccessInformation, which includes the specification of data format, and parameter descriptions that are included in the SPASE information model are important information for enabling accessibility and usability of the data. SPASE descriptions of observational datasets registered on the SPASE registry can be found by perusing the SPASE resource landing pages posted at https://hpde. io/. For example, the landing page of the SPASE description for the fast-mode (4. 5 s) electron phase space distribution data obtained by the dual electron spectrometer (DES) of the fast plasma instrument (FPI) on the number 2 spacecraft of the Magnetospheric Multi-Scale (MMS) mission is posted at https://hpde.
io/NASA/NumericalData/ MMS/2/FastPlasmaInvestigation/DES/Fast/Level2/ Distribution/PT4. 5S. html and displayed in Fig. 10. From the landing page, one can find that the ResourceID of the dataset is simply given by the landing page url but with ''https://hpde. io/" replaced by ''spase://". As discussed in section 3. 2, The URI of the ResourceID, or the landing page url, of a resource essentially reflects the unique path by which the resource is found on the SPASE registry. Using the AccessInformation provided in the SPASE descriptions, all SPASE-registered resources can be uniquely identified and found. The description, Informa-tionURLs, and other metadata on the landing page help the user understand the dataset.
Higher-level processed datasets Sections 4. 1. 1 and 4. 1. 2 focused on the digital resources obtained directly from instrument measurements and simulation models, respectively. Higher-level data products can also be derived from instrument data when researchers apply an additional layer of processing or when data from different instruments or platforms are combined to support a specific study or type of analysis. These datasets are also important in advancing heliophysics and thus require the same capabilities as observational and modeled data, namely for users to uniquely identify, retrieve and understand the dataset. We discuss two examples in the subsections below. 4. 1. 3. 1. Composite datasets and SPASE description strategies. Composite datasets, which may contain data from different sources and instrument types, are becoming more common.
Cutting edge Heliophysics research studies often require data assimilation, for instance, combining observations and modeled data. Solar wind data such as those available from OMNIWeb  (King and Papitashvili, 2005 ; also see https://omniweb. gsfc. nasa. gov/) represent a classic example of a composite dataset. OMNIWeb is available from 1963 onward and combines spacecraft ephemeris data, solar wind plasma data, and interplanetary magnetic field data from the ACE, Geotail, IMP 8, and Wind spacecraft along with geomagnetic indices and energetic flux measurements from geostationary satellites. Also, almost all space physics journals now require authors to submit the data analyzed in support of published research. It is therefore important to have a way to describe these composite datasets to ensure that they are also FAIRcompliant.
SPASE descriptions of both simple and composite datasets are best exemplified by the NumericalData resource (see section 3. 3), as shown in Figs. 3  and 8. That said, the same rationale outlined below also applies to the SPASE Catalog and DisplayData resource types (see left column in Fig. 3 ). SPASE 2. 5. 0, the current version of the SPASE schema (https://spase-group. org/data/model/index. html), includes a new resource type (destined to the data domain in Fig. 2 ) called Collection, which has been added to accommodate the need to describe datasets containing data from multiple, disparate data sources. While we refer the readers to the SPASE model dictionary in general, the SPASE dictionary defines a Collection resource in part as: ``An aggregation of resources, which may encompass collections of one resourceType as well as those of mixed types.
A collection is described as a group; its parts may also be separately described. All the resources that are part of the research effort can be described as a Collection. " 4. 1. 3. 2. Datasets for data science or machine-learning analysis. Artificial Intelligence (AI) and Machine Learning (ML) have become powerful tools in the space physicist's analysis toolbox to assist in scientific discovery, pattern recognition, feature importance, and prediction tasks. Search and access are prerequisites to data science analysis. Metadata and information models are the structure to the information in our world. They give us a way to navigate the deluge of 21st century digital life and permit discovery. All scientists, engineers, researchers require an understanding of metadata and information models.
However, a bottleneck in AI/ML applications is in the preparation of the data for the algorithms, which requires gathering data from various archives, understanding the quality of each data point, aligning them spatially, temporally, or both, potentially rescaling or normalizing, and Fig. 11. A screenshot of a landing page for an example simulation output showing a variety of metadata recorded for the output. At the bottom of the page are two links labeled ''Request output data as a single archive file", which takes the user to an online form to complete the request, and ''Browse output data", which will allow the user to browse the files in the simulation output once the feature is fully developed (See section 4. 2. 4 for more details). structuring the integrated dataset for ingestion into the chosen algorithm.
While there are numerous space physics examples that describe the process (e. g. ,  McGranaghan et al. , 2018b McGranaghan et al. , , 2021b;; Sadykov et al. , 2021)  and provide guidance on the development of the metadata that would be needed to identify and support the searching for or discovery of these datasets, there is not yet a metadata model that can completely and uniquely describe AI/ML-ready datasets. Considerations are being made by the SPASE Group and the SMWT for future extensions of the SPASE model schema to enable descriptions of the growing number of these datasets. More discussion on challenges related to AI/ML datasets metadata are provided in the future outlook section (5. 2).
, 2007) , and virtual wave (VWO)  (Fung, 2008; 2010)  observatories. The virtual solar observatory (VSO), however, does not use SPASE  (Hill et al. , 2009). All the SPASE metadata resources of the early SPASEbased VxOs were hosted in a single SPASE registry residing on Github at https://github. com/hpde/. Lessons learned from the pioneering VxO implementations and the availability of more SPASE descriptions of digital resources based on the more recently released and more matured SPASE model have enabled the heliophysics data portal (HDP) mentioned in section 2. 2. 2 and helped spur the recent development of the Heliophysics Digital Observatory (HDO; https://msqs. gsfc. nasa. gov/hdo/public/) by consolidating and streamlining the previous VxOs  (Fung et al. , 2021; 2022). Fig. 12  shows the general layout of the information architecture, illustrating the relationship between the users, the middleware, and different (distributed) sources of digital resources (e.
g. , data). The green box is the same as the one in the heliophysics information flow diagram in Fig. 9. The middleware, consisting of a query builder and the search engine, communicates with the users via paths (  1 ) and (  2 ) in Fig. 1  and with the data sources via path (3). Direct communication using web services through applications programmer interfaces (API) would take place via path (3a) in Fig. 1. It is important to recognize here that only transfer of metadata is needed along (1), (  2 ), (3), and (3a), making the tasks performed by a middleware much more manageable and efficient even over the internet, and the technical approach more focused and intuitive. If SPASE descriptions of datasets need updating or changing, such as due to reprocessing or relocation of the data, the same middleware would still operate seamlessly.
Detailed descriptions of data products, which become DOI landing pages as DOIs for the datasets are minted, are always available. Access methods from services for all available data products, including web browser tools, and web service methods such as HAPI, are all directly provided to the extent they are offered by the providers. In many cases, browse product plots can be produced directly from an HDP page. One important element within a SPASE description is the AccessURL (Fig. 3 ), which lists the access protocols and services available for obtaining the data via path (4) in Fig. 1. This allows SPASE-based search portals such as the one depicted in Fig. 13  to offer immediate connections to the data discovered in a search. For example, if the AccessURL points to a human-focused web page, users can visit that page and download data through that page.
Most observational datasets are also available through direct HTTP or FTP download (directories of files exposed online). Some AccessURL entries point to an API access mechanism, i. e. , something intended not for direct clicking but for programmatic access via client software that understands the API (see section 2. 2. 4. 4). The NASA SPDF offers the Coordinated Data Analysis System (CDAS) Web services (see section 2. 2. 4. 4 and Table  2 ), and they also offer client software that users can run for creating programmatic or automated downloads (Table  2 ). Another API-based access mechanism is the Heliophysics Application Programmer's Interface (HAPI)  (Weigel, 2021b ; HAPI version 3. 0. 0 Specification, https://doi. org/10. 5281/zenodo. 4757597), which is also a COSPAR Recommended access protocol standard (see COSPAR, 2021) for serving time series data.
A HAPI server has a URL endpoint to describe the datasets it offers (a ''catalog" endpoint), and another endpoint to list the parameters available within each dataset (via an ''info" endpoint that takes a dataset id). The SPASE AccessURL for a dataset accessible via HAPI would list the HAPI server URL and the dataset name, and then any software that knows how to use the HAPI protocol can request and obtain digital data for that dataset. HAPI servers provide data using a very simple streaming format which can be conveyed as simple CSV, or servers can optionally offer JSON or binary versions of the streamed data. A sample HAPI request for data from an example organization has the following standard form: https://example. org/hapi/data?id = DATASET_ID&st art = 2020-01-01 T15:35:00. 123Z&stop = 2020-01-05 T00 :00:00Z.
The result from a valid URL will be a stream of data covering the entire time range requested. If the time covers data from multiple files, it is the server's job to stitch this data together and serve it as a continuous stream. By hiding these arbitrary file boundaries, the HAPI standard offers a simple, conceptual way to think about time series datasets as collections of parameters. In fact, the goal of HAPI is to capture the lowest common denominator of what is needed to serve time series data, making it easy for data centers to add HAPI alongside existing services. HAPI has been implemented by large and small data providers. A list of active HAPI servers is available at https:// hapi-server. org/servers. HAPI and SPASE are independent but interact in important ways. SPASE focuses on data description and enables discovery, while a HAPI server is focused on access -providing a standard computer-based protocol to get to the numbers.
HAPI metadata is minimal and focused on what is needed to scientifically interpret the numeric values. HAPI allows additional metadata (in ways that will not conflict with required HAPI keywords), and it has an optional keyword that can be a link to more rich metadata such as a SPASE document. A search engine for data that uses SPASE lets users find data, and then users can employ other programs in their workflow to obtain data through HAPI (or CDAS, or any other protocol listed in the Acces-sURL element). Many clients and software packages (e. g. in the PyHC) already understand HAPI, and given a toplevel HAPI server URL, can let users explore the contents of a HAPI server and load data for user-selected time ranges. Autoplot  (Faden, 2010)  is one existing HAPIenabled analysis program outside of the PyHC, and Fig.
14  shows an exploration window where a user can select data to plot. The figure also shows how multiple data sources are easy to combine in one tool since that tool can talk to any HAPI server. There are also IDL, Python, and Matlab codes that can read HAPI data, so users can incorporate data reading into their own workflow. Das2 is another access API that can be listed in the SPASE AccessURL element. It is a second-generation web-based data delivery, visualization, and analysis system from the University of Iowa  (Piker et al. , 2018). Having various Java libraries and client-side applications that support space science data visualization such as interactive spectrograms, overlays, waveform plots, etc. , Das2 has more features than HAPI, making it both more powerful and also somewhat harder to implement (https://das2.
We can make SPASE registrations for the repositories of such event lists, e. g. , the Heliophysics Events Knowledgebase (HEK, https://www. lmsal. com/hek/), and bring the list descriptions into the same SPASE database as the event lists that follow the HPEvent format. At the level of simply registering the Event Lists, the current SPASE Catalog Resource is sufficient, with AccessURLs that point to the archived lists of whatever sorts; current lists in HDP are highly varied in this respect. In the longer term, we would like to have tools that operate on Event Lists, aiding in finding overlaps or analyzing characteristics of different classes of events, but to make this possible across Catalogs will require at least a mapping from HPEvent to HEK. The HEK ''VOEvent"-based scheme is more flexible than the HPEvent approach which, while very efficient for simple time series, does not have the generality of the VOEvent approach  (Seaman et al.
For example, users can search active regions, CMEs, filaments, and other solar phenomena for any time interval through the NASA web application Helioviewer. org or the ESA advanced open source software JHelioviewer (https://www. jhelioviewer. org) because it includes the HEK. Some solar and solar wind events are also searchable through the Heliospheric Cataloguing, Analysis and Techniques Service (HEL-CATS, https://www. helcats-fp7. eu/index. html) and the Coordinated Data Analysis Workshop (CDAW) Data Center (https://cdaw. gsfc. nasa. gov/). The SOLAR VO also allows searching across the HEK and has the capability to search instrument data that overlap with the events from the HEK (https://solarnet. oma. be/web-client/hek_events). Using the SPASE metadata standard, including the HPEvent format, will simplify the aggregation of similar datasets from different data sources and event lists to support statistical studies.
, 2021b). By searching by the keyword 'particle precipitation', a researcher might discover datasets that might otherwise not have known about, and thus expand the possibilities for scientific advances (e. g. , Fig. 15 ). In this way, the rich and uniform descriptive capability of SPASE could help lessen the complexity of data wrangling and convergence needs for rigorous space physics scientific research across the landscape of heliophysics digital resources, which requires the use of many heterogeneous datasets to piece together a system-level understanding.
The Ambient Solar Wind Validation Team embedded in the COSPAR International Space Weather Action Teams (ISWAT; https://www. iswat-cospar. org/) initiative was formed to develop an open platform for the community to more easily compare ambient solar wind models in the community, as conceptually represented by the red oval region in Fig. 9. The platform is actually integrated into the CAMEL tools (https://ccmc. gsfc. nasa. gov/tools/ CAMEL/) framework of the CCMC and is publicly available online via NASA's CCMC  (Rasta ¨tter et al. , 2019). A critical backbone of this community effort is the metadata architecture to ensure rigorous documentation of the model settings and reproducibility of the solutions. The team proposes eight different types of metadata that are necessary to register the models in their database.
Data visualization As a use case example of how SPASE information is used to support data visualization (section 2. 2. 5), we point out that SPASE parameter descriptions are used in the Automated Multi-Dataset Analysis tools (AMDA; https://amda. cdpp. eu from the CDPP, see Genot et al. , 2021) to access the actual data in its Plot Manager. The SPASE XML files are stored in a registry and used to display information at each level in the Workspace Explorer of the tool (Fig. 17 ). SPASE metadata are also displayed as titles for axes on the plot (built from Name, Units, Structure, CoordinateSystemName elements of SPASE Parameter container). A user selects a physical quantity or a parameter in the Workspace Explorer that provides SPASE-based information (ResourceName, Description, Acknowledgement, Contact, InformationURL, Tempo-ralDescription, Caveats,.
irap. omp. eu/service/hapi). This will enhance the visibility of AMDA and its datasets via a facilitated access, e. g. , plotting AMDA datasets in Autoplot (https://autoplot. org/) is now straightforward (Fig. 19 ). In general, software analysis and data visualization tools do not use SPASE metadata with perhaps the exception of RenderingHints (under Parameter description; Fig. 8 ) that describes the attributes to aid in the rendering the display of parameters, e. g. Autoplot in Java and pySPEDAS's PyTplot component in Python (https://pytplot. readthedocs. io/en/latest/index. html). However, the CCMC is planning to include SPASE metadata in Kamodo's plotting and display output in the future  (Ringuette et al. 2023).
Open science There is an increasing call for ''open science" within the Heliophysics and most other science communities  (Ramachandran et al. , 2021; Gentemann et al. , 2021). The idea is to make any research result easily accessible and reproducible by other researchers, leading to more collaborative, efficient, and even more rewarding results. This requires that, in addition to the reasoning steps captured in typical journal articles, any data underlying the arguments and any software needed to carry out the analysis must be reported as well. Here, the SPASE metadata and associated DOIs as described in sections 3. 1-3. 5 will be extremely useful. Specific datasets and registered software can be cited in papers by using the SPASE Software Resource (Fig. 2 ). DOIs can also be generated for collections of data and software needed for a paper (section 4.
All these factors indeed impact reproducibility. Having recognized the importance of reproducibility, recent missions, such as Solar Orbiter, have started to tackle this issue by storing and providing access to all versions of their data. Version information will be critical for such comparisons, and needs to be kept in the files' metadata to eventually be reflected in SPASE. Enabling reproducibility of a given analysis is a complex issue involving numerous entities and processes beyond those describing or hosting datasets as focused on here, yet we defer full exploration of this issue to the community for a case-bycase discussion, including the process by which science in the Heliophysics community can be performed in the open from the beginning as required by open science. Such discussions are beyond the scope of this work.