dataesr/scientific-paragraphs-categorization huggingface dataset:
In particular, the association of template bricks with terms precisely describing the concepts encountered in molecular networks paves the way to the development of tools based on the annotation of SBGN maps. Such an annotation is for example a prerequisite for the semantic validation of maps, or for their merging. In this regard, BKO could be used to improve map curation workflows and tools, by for example allowing automatic detection of misrepresentations or repetitions. In addition, extending BKO with sets of bricks expressed in other standard languages such as BioPAX will allow the development of new converters grounded in a formal framework. Such an extension will also open up the possibility to integrate the annotation process we used to analyze the ACSN and PANTHER databases into more systematic workflows  [19]  while using such model repositories as BioModels  [4] , Physiome Model Repository  [28]  or Reactome  [10].
B. Absence. The template brick (BKO:0000492) represents the term "passive transport" (SBO:0000658). The template simple chemical with label "!ATP!" will match the absence of a simple chemical reactant with label "ATP", and the one with label "!ADP!" the absence of a simple chemical product with label "ADP". Hence the instance is not matched, because it consumes and produces such simple chemicals. activity (GO)). Finally we completed BKO by analyzing real SBGN maps taken from the ACSN and PANTHER databases. BKO was built using Protege  [24]  and the owlready2 Python library  [20]. Terms and template bricks were added in the ontology as classes, and the different categories as individuals of a unique category class. Associations between template bricks and terms/categories were added in the ontology as class annotations.
Pattern matching in the maps of the ACSN and PANTHER databases Each map of the ACSN  [17]  and PANTHER  [32, 22]  databases was converted from the CellDesigner format (celldesigner. org/features. html) to the SBGN-ML format  [3]  using the cd2sbgnml converter  [2] , with a posteriori addition of active and inactive state variables to macromolecules and complexes, and of units of information with label "ct:mRNA" (resp. "ct:gene") to mRNAs (resp. genes). Each map was then stored in a Neo4j database (neo4j. com) using stonpy v0. 1. x (github. com/adrienrougny/stonpy), a new Python 3. x version of STON  [33]  specifically developed for easing the identification of instance bricks in SBGN maps. Each PD template brick was then expressed as a Cypher query (the query language for Neo4j graph databases) following the matching rules introduced earlier, and was run against the database containing the maps.
Conclusion The concept of SBGN bricks has been employed for the development of multiple tools including format converters and template-based functionalities in editors. This shaped a demand in an extended and better-organised set of SBGN bricks for providing better support for such tools. We introduce the BricKs Ontology (BKO) that includes a new set of 476 bricks hierarchically organised. BKO is downloadable and browsable online at sbgnbricks. org, and is available under a CC BY 4. 0 license. The completeness of this updated set of bricks was evaluated by checking them against the maps of the ACSN and PANTHER pathway databases. All processes and activities could be accurately identified and annotated, suggesting that BKO may be a valuable resource for the description of recurring concepts in molecular networks.
Classification of all instances of the ACSN and PANTHER databases that were only matched by template bricks representing generic concepts. All bricks that were matched by one of 30 template bricks representing generic concepts (e. g. an irreversible process, a catalytic activity) without being matched by template bricks representing more specific concepts (e. g. a protein phosphorylation, a protein kinase activity) were classified in one of the following six categories. Misrepresentation: the process or activity is badly represented (e. g.
Monitoring programs in urban drainage systems generate, potentially, a huge amount of data from sources distributed in the urban environment, working at relatively high sampling rates for extended periods of time. Collecting data using adaptable and reliable communication systems is the first challenge. Then structuring the collected data is a first requisite for effectively managing the quality and accessibility of the data. In adjacent fields of research, the topic of managing huge collections of data has resulted in several (open) standards and protocols for database structure, transfer and storage to ensure unambiguous definitions on which parties can build their workflows/software. This chapter describes relevant approaches for urban drainage and stormwater management systems, and appropriate standards along with examples from case studies.
2. 5) to be able to capture the dynamic behaviour of an urban drainage system, and over long periods of time of the order of many months, data sets can quickly become overwhelmingly large. The challenges of data communication and storage are therefore not losing any information and ensuring that the data can be efficiently accessed and used, whatever that use might be (data analysis, calibration of numerical models, publication, etc. ). This requires a carefully thought workflow from the sensor to the database, and appropriate strategies for data structuration and storage. Another challenge is that there is also increasing pressure from public authorities to make data available for the public. For example, the new European Open Data Directive (EU, 2019) requires from the Member States that all data produced by public sector activities (including research, ministries, state agencies and municipalities) should be available with open access or at marginal costs, and should be reused as much as possible, including for economic purposes.
This goes along with the concept of FAIR data that is now becoming a major objective of data management in research and many public administrations  (Wilkinson et al. , 2016). FAIR means Findable, Accessible, Interoperable and Reusable. These are the conditions that must be met in order to make data available in a long-term perspective and taking advantage of the current advanced technologies (cloud storage, webservices, big data). The development of a communication and storage strategy should therefore also take that into account. This chapter presents the current state of the art and guidelines for building a strategy and choosing the appropriate tools for data storage and communication, from in situ sensors to interoperable databases. It also provides practical case studies with recommendations from practitioners.
FROM DATA FILES TO STRUCTURED DATABASE 5. 3. 1 Principles and advantages of relational databases Data are usually retrieved from the field as unstructured files, either text or binary files. They can be stored as such on a personal computer, or in a shared repository of the organization or company intranet, if they are to be shared between several users. It is common that data are imported in Microsoft Excel: it is a widely available software that most know how to use. Embedded calculation formulas, programming functionalities and automated graph generation allow the user to get a first insight into the data without too much effort. However, these solutions, even if they are certainly widely used, present major drawbacks: • Apart from very small datasets, the data have to be structured manually: the files are organized in several folders, and/or in worksheets in the case of a workbook.
One must be very rigorous to keep the structure as the dataset grows and becomes more complex. Eventually it can become impossible to maintain. • In some cases, the data files do not contain all the information required to use the data, such as variable names, units, time zones, contact of data producer, types of sensors and so on. These metadata must be added to the dataset, thus adding complexity for manual maintenance, particularly when several people are involved in data collection. • Access to the data themselves is difficult and slow: one has to find the right file in the right folder, open it or import the data into a processing software without making errors with the format and so on. • There is no possibility of sharing the data while ensuring its integrity and consistency: copies of the file can be multiplied in several places, files can be modified, updates are not necessarily transmitted.
These limitations can be overcome using relational databases. In a relational database, information is organized in bidimensional tables (or relations, with columns and rows) that are related to each other. Each record in a table (i. e. each row) can be identified individually using a unique identifier (usually an integer value) that is called a primary key. The connection between tables is achieved using foreign keys. It consists of adding a column to a table containing the identifier for each row of the related row in the other table. All complex relations between tables can usually be decomposed into simpler ones. The main advantages of relational databases compared to unstructured files are: • Information is not often duplicated, thanks to the use of primary and foreign keys. Thus, there is no information redundancy.
The data volume is lower, which optimizes storage. • The structure of the database is fixed at once, which ensures a better consistency of data and a better sustainability of the database, as the dataset grows over the years. There can be constraints on the data (e. g. one value per date for a given time series, to ensure that there are no duplications). • Relational databases use a specific programming language for accessing the data: SQL (Structured Query Language). It is a standardized language that allows quick and efficient access to the data, whatever the size of the dataset, which can be unlimited. Complex manipulations of data can also be carried out with SQL. All these functionalities are implemented in dedicated software tools called Data Base Management Systems (DBMS). Most of these tools come also with additional functionalities such as: • Data centralization (on a server) and management of access rights (data integrity/security).
Existing DBMS and software solutions Several Data Base Management Systems are available on the market. A short list of the five 'best known' DBMS is presented in Table  5. 1. What distinguishes the software tools are mostly the licence conditions (proprietary vs. free and open-source), the maximum size allowed for the database, and the possibility of adding spatial data into the database. This spatial extension feature makes databases compatible with GIS (geographical information system) software, such as QGIS or ArcGIS. GIS software can use a database with a spatial extension as a centralized data source. The database can then benefit from the functionalities of the GIS software (data visualization, maps and so on). Microsoft Access is distributed with the Microsoft Office Suite, and as such it is the first DBMS that one may come across.
It is suitable for small datasets (due to the size limitations) that do not need to be shared between several people. It has no server service. This means that each database is one file stored on a personal computer with no access control: it can be duplicated or modified by any user of the computer. MySQL is the backbone behind most of the commercial websites in the world. PostgreSQL and OracleDB are more specialized in scientific applications. The main difference between them is the licence conditions. PostgreSQL is the open-source reference for structuring and management of environmental observations. It is used for the storage of data from many research observatories throughout the world, as for example in BDOH that stands for Base de Données pour les Observatoires en Hydrologie (i. e.
Database for Hydrologic Observatories,  Branger et al. , 2014) , AMMA-CATCH  (Galle et al. , 2018) , ILMS  (Zander et al. , 2013) , Hydroshare  (Heard et al. , 2014; Yi et al. , 2018)  or Drought Observatory  (Magno et al. , 2018). All the DBMS (even those with a server service) can be installed on a regular PC (which then becomes a 'local server'). One can then benefit from the robust data structure that they offer, the unlimited size of datasets and the efficiency of the SQL queries for data manipulation and access. However, these systems show their full potential (in particular data sharing with access control, proper handling of multiple users simultaneously and data backups) when installed on a proper server independent from the PCs of the data collectors and users. Data users can have access to the database through specific applications installed on their own computers (heavy or rich client), or more simply with a web browser (thin client).
A typical architecture is called 'three-tier' and involves the data server (that contains the DBMS and the datasets), a dedicated web application (program that accesses the server), and a web server on which the users can connect with their computer browser. Commercial software dedicated to environmental monitoring and based on DBMS can be found on the market. Aquarius by Aquatic Informatics (https://aquaticinformatics. com/products/) and WISKI by Kusters (https://www. kisters. net/NA/products/wiski/) can be cited as examples. Both rely on Oracle DB databases. These software products provide solutions for data structuration, backups, secured access plus a wide variety of functionalities for data visualization and processing. There are also a few free and open-source alternatives made available for a wider public, such as BDOH and Hydroshare (see Section 5.
Typical data structuration for environmental time series As an alternative to pre-packaged commercial or open-source solutions, it is also possible to develop custom-made databases. This requires more time and human resources for the informatic development, but it ensures that the database will be adapted to the specificities of the data and to the needs of the end-users. CUAHSI (Consortium of Universities for the Advancement for Hydrologic Science, a non-profit organization that includes many American universities) proposes guidelines about important aspects for database development, such as defining a controlled vocabulary for the names of the variables  (Horsburgh et al. , 2014)  and a database structure that can be used as a template  (Horsburgh et al. , 2016). Since most data collected are numerical time series, they must be also supported by metadata, i.
e. additional information about the measurement attributes (names, time zones, locations of sensors, physical conditions at the measurement sites, etc. ). These metadata are essential for the interpretation and further use of the data. Chapters 7 and 9 highlight the type of metadata to be collected and their importance for data quality assurance, respectively. An important step when building a database is the design of the data structure, i. e. how to dispatch the information into several tables, and how these tables are related to each other. Several options are possible, according to the specific nature of the data and the objectives of the data collectors and users. For example, if there are several measurement points scattered over a large territory, it will be particularly important to be able to locate these measurement points.
In other cases, emphasis can be put on the sensors that produce the data, keeping track of information such as the nature of sensor, brand, series number, and maintenance history (sensor replacement, etc. ). In other cases, the database will have to be able to deal with several possible time steps for the time series (or even variable time steps). For example, in the BDOH database  (Branger et al. , 2014  see also Section 5. 5. 1), one of the main requirements was to be able to manage time series with variable time steps. Another important feature was that each value should be associated with an estimation of data quality and possibly a confidence interval corresponding to the data uncertainty (see Chapter 6 for uncertainty assessment). The database was designed with one single table containing all the measured values (by end of 2020, it contained over 60 million entries), that is represented in Figure  5.
Supply of information to databases In addition to choosing the appropriate tools and building the database, one crucial point is also to define a processing workflow that ensures that the data actually get into the database and are updated regularly. It can be a fully automated process from in situ sensors to the database server and even to the data online publication, or it can include several manual steps such as data collection, data validation, uncertainty assessment, etc. All of these aspects must be taken into account and organized so that no data are lost and the information system is efficient  (Horsburgh et al. , 2011). One should ensure that the time between actual measurements and the proper storage of the corresponding measured values in the database must not be too long. A delay of 3 years between data collection and storage is not acceptable, although it is unfortunately not that uncommon.
DATABASE INTEROPERABILITY 5. 4. 1 Definition and interest Interoperability can be defined as a set of characteristics of a system, that allows it to exchange information seamlessly with other systems. In the world of data management, interoperability means that several databases can communicate with each other and exchange information on the data they contain (metadata) and/or data itself. It is one of the pillars of the FAIR data concept. Interoperability presents huge advantages for data management and exploitation. Instead of having to build and maintain one large and centralized database that must contain everything, it allows entities to build several smaller and specialized databases, that are easier to design and maintain, especially if the data they contain are produced by different entities.
It can also bring together data of different sorts that come from various sources, thus providing additional elements for data interpretations. A typical example could be bringing together maps of the sewer system and stormwater control measures (SCMs), logs of the operations on weirs, and continuous flow measurements from in situ sensors. Mostly these three types of data are not available together, at least not in a straightforward way. One person has to request them from two or three different people, each time a question arises. A change of direction of flow monitored by the sensors could typically be interpreted after several weeks of investigation as due to an operation on a weir in a nearby branch of the sewer. If these data were stored in interoperable systems, they would be known to all the systems (and operators) as soon as they are uploaded in their respective databases, thus saving a lot of time and energy.
Finally, interoperability is the cornerstone of public data portals, such as Data Grand Lyon (see Section 5. 5. 3) or the Theia/OZCAR Information System  (Braud et al. , 2020). Such data portals present and give access to datasets that come from various data producers and are stored in various databases. They are the full implementation of the FAIR data concept, and as such, can be considered as the future of data storage.
Interoperability standards and examples As for sensors, interoperability means that communication protocols must be defined for databases. These communication protocols operate through the web, and as such are called webservices. In the field of environmental data, the Open Geospatial Consortium (OGC) is in charge of defining open standards for these webservices. The OGC is an international standards organization, that was created in 1994, initially to foster the development of open geospatial tools. The most well-known and widely used standard webservices proposed by the OGC are the CSW (Catalogue Service for the Web) that is behind online metadata catalogues (such as https://www. data. gouv. fr/fr/ (accessed 07 Dec 2020) in France, https:// www. ukdataservice. ac. uk/ (accessed 07 Dec 2020) in the UK, etc.
More specific to hydrological data, the WaterML standard (https://www. ogc. org/standards/waterml (accessed 07 Dec 2020)) has been defined more recently. It introduces additional concepts specific to hydrological time series, such as gauging and rating curves, or temporal data interpolation. However, it is not as widely used as the three other standards presented here. OGC provides standards and specifications, but does not provide ready-to-use tools. The implementation of the standards has to be carried out. The standards are long and complex, they do not have to be implemented to their full extent. Several implementations of the SOS webservice are available. The most widely used implementation, recognized as reference, is the open-source German-based 52°North solution (https://52north. org/software/software-projects/sos/ (accessed 07 Dec 2020)).
Practical recommendations Data portals that implement interoperability are high level software constructions. As a water scientist or a practitioner, the objective is not to be able to implement them or manage them directly, but just to understand the underlying concepts enough to be able to communicate with the specialists. The development and maintenance of interoperability in databases requires skilled software developers and IT support. However, many organizations now do have the corresponding human resources, in universities or research institutes, and in local authorities' administrations (regional councils, city councils). Although we can see interoperable data portals as the future of data storage, we must also keep in mind that the data must be properly structured and stored beforehand, following the guidelines given in the previous sections.
Structure and main features BDOH is based on open-source technologies such as the database management system PostgreSQL/PostGIS, and a web application developed in PHP and Javascript. The structure of the database considers stations that are points in space where physical quantities are being measured. The time series for each of these quantities are stored at variable time steps. Each value is associated with a date/time, a code appreciating the quality of the data (see also Section 9. 2. 2), and an estimation of the measurement uncertainty (see Section 8. 4). Functionalities for data managers include data import from flat files (text files) in various formats, automatic calculation of time series derived from other time series (for example streamflow calculation from water level measurements using rating curves, and taking into account rating curve changes over time, or contaminant fluxes, see Section 9.
Advantages and drawbacks The use of a relational database management system and a web application means that the data is easy to browse and to extract, and is permanently available. It is also hosted on a server with automatic backups and security checks so that the integrity of the data can be ensured. The qualification of the data is also an asset, although BDOH does not provide a method or workflow on how to assess this quality. BDOH also fostered exchanges and general improvement regarding data production practices. The main drawback is that of course it is quite some work to install and set up a BDOH instance. It requires computer resources and skilled human resources, in particular an IT system that can manage a Linux server. Otherwise, the time required for maintenance and data administration is quite low, approximately a 10% part-time job for over 60 million records.