dataesr/scientific-paragraphs-categorization huggingface dataset:
Collection of users' needs for the Theia/OZCAR IS Regarding expectations towards the Theia/OZCAR IS, the "Tour de France" showed that users were primarily interested in searching data through variable names, spatial location and time slots. The organization of the data into datasets was useful for the producers themselves, as datasets gather coherent sets of co-localized variables, but they are often recognized as hardly understandable by external users. Users were interested in metadata, but once identified, they wanted to go further and get the data itself with a unique extraction format. Data producers were also reluctant to manually fill files with information that they have to provide several times in different formats and were expecting automatized procedures for filling metadata. A previous experience -an effort to develop a metadata catalogue for the hydrological observatories of OZCAR-RI -was also often cited as the "example not to follow".
This metadata catalogue  (André et al. 2015)  was conceived to respect the INSPIRE norms and was able to harvest A C C E P T E D M A N U S C R I P T existing ISs, when the ISs were providing the required web services. For the observatories that were not providing the required services, a manual system was proposed to feed the metadata. However, the usefulness of the data portal remained limited due to the heterogeneous definition of the granularity of the datasets that was not harmonized. Moreover, the names of the datasets were not self-sufficient to understand what they contained. Metadata that were not automatically harvested were quickly obsolete, and metadata documentation was incomplete, implying that access to the data portals was not guaranteed. Users also complained about this catalogue having been built without consulting the future users and not responding to their needs, although the portal was achieving a high technical and interoperability level.
In order to refine the Theia/OZCAR IS functionalities, three types of user were identified: scientists that will use the data portal, data producers and funding institutions. Working groups were organized in April 2018, during the annual OZCAR-RI meeting (40 participants), to elicit the needs for the different groups of users. Each group had to answer the questions that are listed in Table  1. To answer them, participants were asked to write their answer on a post-it, without communicating with others. Then, they were asked to read their answers that were put on a paperboard by one facilitator and grouped when they were addressing similar needs. Another facilitator was in charge of taking note of what participants were saying. At the end, the facilitators of both groups presented syntheses of the main outcomes of the groups in a plenary session.
2011)  and the responsibility of the data quality and dissemination had to remain in the observatories. Thus, it was chosen to focus on interoperability with existing systems, so that metadata (and data in a near future) could be accessed transparently, whatever the data location, the local IS and data storage choices (e. g. Ames et al. 2012). This principle allowed subsidiarity among observatories. The strength and applicability of a subsidiarity model for developing IS between the central and the local level has largely been demonstrated  (Salvemini, 2010). To interface the existing distributed observatories' systems with the Theia/OZCAR IS, it was proposed to set up an information flux. For each observatory, this flux consists of pushing their metadata and data towards the Theia/OZCAR IS in a format implementing a common data model.
This so-called "pivot data model", was built based on standards to have all the necessary information to (i) handle different types of data; (ii) make the data FAIR; and (iii) ensure the functional requirements of the system, such as the search criteria in the web interface and the future implementation of standardized web services. This step is shown with the red rectangle A C C E P T E D M A N U S C R I P T in Fig. 2. Note that to achieve data exchange, each observatory had to develop a script that extracts the information to be transferred from their existing IS to the central Theia/OZCAR IS in a format respecting the pivot data model (see details in Section 3. 2). The Theia/OZCAR IS was designed to provide services to users or interoperable systems. This includes data discovery, exploration and access functionalities to users through a web interface; standardized metadata/data exchange services actionable by machines (e.
g. harvestable catalogue service); services that provide the metadata files compliant with the DataCite Metadata Service Group (2019) that can be transmitted to data producers or pushed towards a DOI registration agency. Users (either humans or machines) consuming these services are represented by the blue rectangles in Fig. 2. The next section provides more details on the Theia/OZCAR IS architecture. The pivot data model and system architecture were built iteratively using the AGILE approach, with frequent exchanges between the development teams, data producers and the future users.
Implementing data discovery in the Theia/OZCAR Information System The current version of the Theia/OZCAR IS only implements a part of the functionalities described in Section 2. 4. They are described in this section and include the definition of a controlled vocabulary for variables names, the definition and implementation of the pivot data model for metadata and data transfer, and the building of the web portal. Currently, the Theia/OZCAR web portal only allows data discovery of time series, and data download is not available yet. The discussion section presents how we plan to complement the Theia/OZCAR IS to implement the remaining functionalities presented in Section 2. 4. A C C E P T E D M A N U S C R I P T.
Defining a controlled common vocabulary Consultation of users showed that their main motivation for using the Theia/OZCAR IS was the ability to find information down to the level of available variables with their associated location and detailed time windows. As a second priority, users cited the ability to know the types of sensors and measurement protocols. The collection of variable names given by the observatories showed different choices of names for the same variable (i. e. rain, rainfall, precipitation, liquid precipitation) in French or English, with more or less precision (daily rainfall, hourly rainfall, rainfall, rainfall at 2 m height). Harmonization of the vocabulary was thus necessary to make the data presented by the Theia/OZCAR IS more discoverable. The use of a hierarchized controlled vocabulary was chosen.
This vocabulary was built from the categories and variable names of the Global Change Master Directory (GCMD 6 ). Launched by NASA in 1987, the GCMD covers subject areas within the earth and environmental sciences. Today it is one of the largest public keywords inventories in the world. Its use is recommended in the French Data Terra IR. As it does not include chemical data, it was enriched using the SANDRE 7 vocabulary for chemical data (the French norm for data reporting for the EU Water Framework Directive) and with additional categories relevant for the OZCAR-RI community such as "surface fluxes" (Fig. 3 ). The controlled vocabulary consists of hierarchized concepts of variable categories covering the different compartments of the critical zone, where variable names are found at the last level of the hierarchy.
A given variable or a category may be related to several compartments (e. g. "surface flux" was related both to "land surface" and "atmosphere", see Fig. 3 ). The decision on which variable name was retained in the Theia/OZCAR controlled vocabulary was made by the scientists of the project team, and data producers were then consulted to validate the choices. Without yet including all the variable names of the ADES groundwater operational portal, the final number of variable names that are listed on the Theia/OZCAR web interface is around 300, with about 45% related to chemical data. Some of these data are commonly measured (in more than 80% of the observatories), while a large part (60%) are measured by only one observatory that can, however, measure it at several measurement stations and in different countries.
In order to promote semantic interoperability of the data, a mapping was carried out that links the terms of the Theia/OZCAR thesaurus with exact or similar concepts of international thematic thesauri (see Table  2  for details of the thesauri considered). This mapping was done for each variable name and category name. In addition, Theia/OZCAR variable categories and variable names thesauri were formally described using the SKOS (Simple Knowledge Organization System) standard 8. They were published 9 using the Skosmos 10 system, an open source web-based SKOS browser and publishing tool  (Suominen et al. , 2015). Since the beginning of the project, some data producers have changed their variable names to GCMD, but they can keep their own variable names in their IS. The mapping between the data producers and the Theia/OZCAR variable names is performed in two steps.
Defining a pivot data model for information exchange between observatories IS and the central Theia/OZCAR IS According to the "Tour de France" and the consultation of users, it was obvious that one single metadata standard would not be sufficient to cover all the system functionalities that were identified. It was proposed to build a common data model based on different standards of metadata, the so called "pivot data model", as the central element of the Theia/OZCAR IS architecture. We considered the ISO19115/INSPIRE and O&M (Observation and Measurement) standards, as they are necessary to set up standardized metadata/data exchange services, and the Datacite 11 standard, as it is used by DOI registration services. We also considered the schema. org standard, as it allows datasets to be referenced by Google Dataset Search, and the DCAT standard, as it allows facilitating interoperability between data catalogs published on the Web.
Technical solutions for building the Theia/OZCAR IS It was decided to implement the pivot data model in JSON (JavaScript Object Notation,  Bray, 2014) , where the geographical features implied in observation, such as the points, lines or polygons are described in the GeoJSON format  (Butler et al. 2016). For the data files, a. csv format was adopted. The choice of the JSON implementation was motivated by the following reasons. JSON is less verbose than XML. JSON can be read and understood by humans. On the Theia/OZCAR IS, this format was easier to parse, and there are tools for validating the format of the files (JSON Schema, IETF 2019). The MongoDB database was chosen for storing the metadata as it is a schemaless database that facilitates its evolution if new data types have to be included.
Moreover, MongoDB database is a no-SQL document-oriented database that is adapted to store, retrieve and manage JSON semi-structured data. The technical tools deployed to build the Theia/OZCAR IS architecture are shown in Fig. 5. Figure  5 (a) illustrates the information flow from the various observatory ISs to the storage system of the Theia-OZCAR IS. Each data producer has to develop a script that extracts and formats the information from their existing IS. Time series measurements must be provided as. csv files and packaged into. zip archives (one per dataset). Data and metadata are uploaded to Theia/OZCAR IS as fat. zip files using the HTTP PUT request method. Once on Theia/OZCAR IS, each data deposit triggers a process that validates the transmitted information and stores the data. The format of the JSON file and the data files are checked for possible errors (using the JSON schema and regular expressions for.
csv files). Validation error details are automatically communicated to the data producers so that they can correct their files. Once validated, the content of the JSON files is imported into a MongoDB database from which it can be processed to feed the web interface or to provide web services. The data files are stored in the file system. This data flux was organized as a continuous workflow. The frequency of update of the JSON files was left to the data producers. When a new version of a JSON file is A C C E P T E D M A N U S C R I P T transmitted to the Theia/OZCAR IS, previous information is deleted and replaced by the new one in the Theia/OZCAR IS MongoDB database. Therefore, the copy of the information available in the Theia/OZCAR IS is kept up to date and can be considered as a data cache.
For dynamic datasets to which a DOI is assigned, it will be necessary to handle versioning of DOIs as recommended by the Research Data Alliance  (Rauber et al. , 2015)  and already proposed by some data repositories like Zenodo 18. (ii) the metadata portal, which is the web user interface for discovering the data. (iv) the thesaurus application, which stores and publishes the Theia/OZCAR controlled vocabulary on the web. Theia/OZCAR controlled vocabulary is formally described in SKOS (Simple Knowledge Organization System). The frontend application (web interface) relies on Skosmos  (Suominen et al. , 2015) , an existing tool developed in PHP, which provides a web interface for browsing the thesaurus, and the Apache Jena triple store database. The application frontends (when there is a web interface) were developed using the Vue.
Web portal interface The design of the web portal interface was done to meet ergonomic criteria such as ease of use, consistency and efficiency. It is a single-page application 19 that offers both a listbased and a map-based search. The users' needs in terms of search criteria, listed in bold in Table  3 , were implemented in the web interface that is illustrated in Fig. 6. The search criteria appear as faceted components on the left and allow search by variable names, spatial location, time windows, geology, climate, funding institutions and observatories, as well as full text search. The data that meet search criteria are presented both on an interactive map as clustered markers, and in a list in a folded table at the bottom of the interface (bottom of Fig. 6 ). The list represents a first level of information with a summarized description including the variable name, the location of measurement, the dataset title to which the variable belongs to, and the data producer.
Data producers found this effort acceptable, given their existing A C C E P T E D M A N U S C R I P T workload. Furthermore, the use of the pivot data model allowed each observatory to retain flexibility in the way they organized their data in datasets (let us recall that the information included in the pivot data model is organized in datasets), and to select the datasets they want to share. Data producers have to keep the information about their data up to date in their IS and they can choose the frequency with which they push their data to the central Theia/OZCAR IS. However, to ensure that the information available in the Theia/OZCAR is up to date, it is advisable that they automate the transmission of the JSON + data files once they do an update on their IS. The proposed system requires a minimum of local IT skills in each observatory to be able to maintain the scripts implementing the pivot data model.
However, in specific cases, this service can be provided by IT engineers who are part of the Theia/OZCAR central IS. In any case, the success of such an initiative requires the allocation of enough human resources for its creation, maintenance and evolution. Therefore, it relies on the long-term support of the institutions. This is, however, rarely the case, as underlined by  Flint et al. (2017). In the design of the Theia/OZCAR IS, a "push" model, in which data producers push their information towards the central Theia/OZCAR IS, was chosen. An alternative could have been to create a register and "pull" model instead. Two options were possible: (i) web services that provide read-only access, like in the CUAHSI HIS that uses WaterOneFlow web services); or (ii) a system in which data producers would give direct access to read their database using solutions like ETL (Extract Transform and Load) or homemade scripts.
However, not all data producers were able to implement the first option. The second option would not have been accepted for security reasons. In addition, this kind of solution would have been more difficult to maintain for the Theia/OZCAR IT team, as each extraction script would have to be adapted to each data producer, as each local database has its own structure and not all of them implement the A C C E P T E D M A N U S C R I P T dataset concept. The "push" solution that was finally adopted required more efforts from data producers at the beginning, as they had to write the extraction script to implement the pivot data model in JSON, but it left them with more flexibility in the way they organized their data. In addition, the collective discussions on the names of categories and variables, on the granularity of datasets and on the metadata gave an active role to the data producers.
They became aware of the need to document the datasets. They often had to enrich the information about their data and realized that this was necessary for a larger diffusion and use of their datasets. They also became aware The inventory also revealed that some observatories were collecting "ecological" data that were not time series, such as species names. This could require extensions of the pivot data model to document properties of the station such as local land cover, depth of sensors, etc. We will explore the FeatureOfInterest O&M concept to accommodate this kind of data. Some OZCAR-RI observatories also collected social science data, like interviews and questionnaires. However, the extension to such kinds of data has not yet been planned, as it would imply data management complexity that is still a research area  (Flint et al.
About the design of the web portal interface The web portal interface was designed based on users' needs and expectations. In particular, users were really demanding a map interface. The design of the Theia/OZCAR interface is quite similar to the HydroShare 21 interface. HydroShare is the latest version of the CUAHSI data portal, which not only includes publication and discovery of data, but also allows sharing of models, for example hydrological models a less convenient faceting system than SOLR but avoids using two different systems to persist the data for ensuring data consistency, and to index the data for efficient search purposes. "Consistency" in database systems means the database is designed such that every read receives the most recent write or an error. The chosen system favours consistency, whereas systems like SOLR or ElasticSearch prefer "availability" (i.
e. such that every request receives a (non-error) response, without the guarantee that it contains the most recent write) to "consistency". So far, the scope of the full text search in MongoDB 23 is sufficient for our needs. Another important point when designing an information system is the speed of responses to queries. The Theia/OZCAR interface was rendered using client computer capabilities, and the MongoDB database was set in replica-set allowing queries to be distributed between different server instances. Unlike the SOLR search engine, this setup allows database resilience but cannot distribute operation of a single query between different server instances. For the moment, only one-third of the expected datasets are provided to the Theia/OZCAR IS, so the MongoDB database only stores a small fraction of the metadata.
It will be important to verify that query performance remains 22 https://en. wikipedia. org/wiki/Apache_Solr 23 https://code. tutsplus. com/fr/tutorials/full-text-search-in-mongodb--cms-24835 A C C E P T E D M A N U S C R I P T acceptable when all of the datasets are included. This will also have to remain true when the datasets will include high frequency datasets that are increasingly common in critical zone studies and have started to be acquired within the OZCAR-RI network (e. g. Floury et al. 2017). A solution to face these issues could be to rely on a search engine on top of MongoDB database. So far, it is not possible to know if the system meets users' expectations in terms of user-friendliness, as the data discovery portal has just been put into production. Available feedbacks that led to modifications and improvements of the web interface came from the scientists leading the project and from scientists and IT teams involved in the building of the Theia/OZCAR IS.
Therefore, user authentication will be necessary to manage access rights to datasets with embargo, or to manage access granted only to some identified users. Authentication will also be required in order to provide statistics about data downloading to data producers, which is a criterion examined by funding institutions. The implementation of the authentication service will be built in synergy with the other French data poles. Technical interoperability will be guaranteed thanks to the implementation of standardized web services (OGC). In particular, they will be implemented to allow interoperability with other data infrastructures such as the European eLTER RI with which OZCAR-RI shares some types of data, or Theia spatial data infrastructure. Such services will allow the harvesting of the Theia/OZCAR catalogue via the CSW protocol and the data provisioning of time series via SOS protocol.
In order to increase the motivation of data producers to share data and the recognition of their work, training will be provided to help them assign DOIs to their datasets, to provide incentives for the publication of data papers (e. g. Nord et al. , 2017; Guyomarc'h et al. , 2019) , and recommendations for licensing of the datasets (e. g. Creative Commons licenses 24 or ETALAB v2. 0 25 , the French reference license). A first seminar, conducted in February 2019, was held to discuss the definition of datasets and their granularity. Recommendations on how to fill the associated metadata were also given in order to encourage scientists to publish their data. system is in production. First contacts were made with CINES 26 which proposes archiving solutions for long-term preservation  (Massol and Rouchon, 2010; Diaconnu et al.
Conclusions This paper describes a methodology for designing a data infrastructure from existing updated. The Theia/OZCAR IS was built using a no-SQL MongoDB database that stores metadata used to respond to users' requests on the web interface. The first part of the project allowed data discovery, and future steps will provide data access and interoperability services. The proposed approach can be adapted to other disciplines and/or data portals with a legacy of distributed data management. The codes and documentation are free and available on Github. The Theia/OZCAR IS will be considered as a success if researchers use it to retrieve the latest versions of their own data. The web portal interface presented in this paper must now be tested by users. As we use an Agile development approach, user feedback will be taken into account to improve and enhance the user-friendliness of the IS.
performed the same kind of mapping for the INSPIRE 11 DataCite: https://datacite. org/ and DataCite standards. The mapping between the various standards is provided in a dedicated data model documentation directory of the project's Github repository 12. The global conceptual schema of the pivot data model is provided in Fig. 4. A complete description can be found in the directory of the data model documentation in the project Github repository 13. The pivot data model includes three main concepts: data producer, dataset and observation. A dataset is a collection of observations. The information describing datasets (what, where, when, who, how, data use and access condition) was based on the ISO 19115/INSPIRE standard 14 and the DataCite standard. These metadata are required to identify the dataset and its "institutional context".