dataesr/scientific-paragraphs-categorization huggingface dataset:
Ozymandias  [23]  is a KG for the Australian fauna that integrates data from several sources, including the Atlas of Living Australia, the Australian Faunal Directory, the Biodiversity Heritage Library and the Biodiversity Literature Repository. OpenBiodiv  [24]  integrates information extracted from the biodiversity literature into a graph database using the OpenBiodiv-O ontology and an RDF version of the Global Biodiversity Information Facility (GBIF) taxonomic backbone. TAXREF-LD  [25]  is a KG representation of the French national taxonomical register for fauna, flora and fungus that interlinks information about taxonomy, species interactions, development stages, biogeography, conservation statuses, etc. In a recent talk at TDWG 2021, Michel et al. [26]  called for more biodiversity data producers to start publishing KGs.
However, for now, building a KG from multiple data sources is a complex and time-consuming task that demands high Semantic Web expertise, and we are not aware of an existing tool specifically designed to help ecologists transform their data sets into interoperable KGswith the notable exception of the iKNOW project  [27] , which is very similar in spirit to our work, but whose current status is unknown to us. In this paper, we present inteGraph, a framework and toolbox that facilitates the process of building a KG from heterogeneous and distributed (semi-)structured data sources in the biodiversity domain. With inteGraph, users can create automatic and reproducible semantic data integration pipelines simply through the provision of configuration files. This declarative approach requires no (or little) code from the user and minimizes the amount of manual effort and Semantic Web expertise required to turn datasets into interoperable KGs.
Data sources Data sources can be (and often are) distributed on several machines, on a local network and/or on the web. Data must be accessible in a (semi-)structured form, for instance as tabular (e. g. tables in relational databases or in CSV files) or hierarchical data (e. g. data in XML or JSON format). At present, inteGraph does not include information extraction components that would allow the integration of unstructured textual data from the literature. In our running example, we will use inteGraph to build a trophic KG from the following three data sources: • The Fun Fun database  [13]  collates fungal functional trait data, including information about trophic guilds, from a variety of sources, for thousands of species across the fungal tree of life. Data are provided in a tabular format (CSV file) and can be downloaded from Zenodo (https://zenodo.
org/record/1216257). • BETSI  [11]  is an open database gathering data on morphological traits and ecological preferences for 7 taxonomic groups of soil invertebrates (Aranae, Carabidae, Chilopoda, Collembola, Diplopoda, Isopoda and Diplotesticulata) from about 2000 literature references. BETSI is accessible on demand via a web portal (https://portail. betsi. cnrs. fr) that provides the user with an interface to write queries and download subsets of the database in a tabular format (CSV file). In the following, we will integrate a dataset containing Carabidae diet data. • The Global Biotic Interactions (GloBI) provides open access to species interaction data (e. g. predator-prey, pollinator-plant, pathogenhost, parasite-host) aggregated from existing open datasets  [32]. As of April 2023, GloBI contains over 17 M interaction records obtained from 342 datasets, covering 823,033 taxa.
GloBI provides several ways to access its data, including a web portal (https://www. globalbioticinteractions. org/), a downloadable snapshot of the entire database in a tabular format (CSV file), and a web API. In our example, we will use the web API to download data about the trophic interactions of Collembola. Information from these three data sources will populate the data layer of our trophic KG once it has been transformed into a common representation.
Target ontology InteGraph adopts a top-down approach to KG creation. In this type of approach, the schema layer of the KGis populated with a predefined ontology. The semantic data integration process then consists of populating the data layer of the KG with data extracted from the different data sources, and creating semantic links between the schema of the sources and the (global) schema of the KG using mapping rules. To reconcile schematic and semantic heterogeneities between our trophic data sources, the schema layer of our example KG will be populated with two ontologies: the NCBITaxon ontology and the Soil Food Web Ontology (SFWO)  [7]. NCBITaxon is a formal translation of the NCBI Taxonomy database into an ontology, in which each taxon is treated as a class whose instances would be individual organisms, e.
g. 'Nicolas Le Guillarme' instance_of NCBITaxon_9606 (Homo sapiens). To our knowledge, the NCBI Taxonomy database is the only taxonomic nomenclature available as an OWL ontology. SFWO is an ontology for representing knowledge on the trophic ecology of soil organisms across taxonomic groups and trophic levels. SFWO captures the semantics of trophic concepts such as trophic interactions, feeding processes, diets or trophic groups. SFWO also includes machine-interpretable definitions for most of these concepts, that allow for inference of implicit knowledge using automated reasoning, e. g. deducing a consumer's diet(s) from the trophic interaction(s) in which it participates.
Triplestore A triplestore is a database management system, i. e. , a software used for storing and querying a database, specifically designed to support the storage and the efficient querying of RDF triples. A triplestore is needed to store both the schema and data layers of a KG. Information stored in the triplestore can be retrieved using SPARQL queries. A multitude of triplestore implementations are available (see Ref. [33]  for a survey), which offer different capabilities and performance in terms of data storage and indexing, query processing, reasoning, etc. InteGraph assumes the existence of a running triplestore instance. It is not tied to a specific implementation and provides connectors to several triplestore solutions. The user is expected to provide connection information as part of the pipeline configurations.
Data extraction This first step of the ETL data integration process involves collecting data from the data source. InteGraph implements a number of components to connect to different types of data sources. At the moment, inteGraph supports the following types of data sources: • File-like data sources: inteGraph can download files from remote or local file-like sources by specifying the local path or the URL of the source in the configuration. Archive files, including compressed archives, are supported, and unpacked before staging. • HTTP data sources: inteGraph can extract data from remote databases exposed through a web-based API by sending HTTP GET requests to the API endpoint. In that case, the user is expected to provide the URL of the endpoint and the query string. Paginated results are supported using the limit and offset parameters.
These two components alone are sufficient to access most ecological datasets. We plan to add more connectors in the future, including connectors to SQL databases, RDF databases, etc. The extracted data are staged on the local file system. Fig. 6a  shows the data extracted from our three example data sources. The three datasets use different data structures and terminologies to organize and describe taxonomic and trophic information. • The Fun Fun database uses the Index Fungorum taxonomic nomenclature. Each line of the data table contains a single trait information for a given taxon. The name of the trait is given in the trait_name column and its value(s) is given in the value column. The terminology used to describe the guild of each taxon is inherited from the FunGuild database. • BETSI does not encode taxonomic information using identifiers from a reference taxonomy.
Data transformation The second step of the ETL data integration process involves transforming the staged data into a RDF graph, i. e. a set of RDF triples. In inteGraph, data transformation consists of two successive operations: data cleansing and schema mapping. Under the term data cleansing, we include all the dataset-specific data processing operations that aim at formatting the extracted data so that they are ready for further processing by the schema mapping component. This includes operations such as removing or filling missing values, removing duplicates, dropping irrelevant data, splitting strings (i. e. splitting a string representing a set of values, e. g. 'bacterivore-detritivore', into a set of strings, each string representing a single value, e. g. 'bacterivore', 'detritivore'), joining two or more data tables, etc.
Fig. 6b  shows an example of applying cleansing operations to the Fun Fun dataset so that each line of the data table contains a single guild value. As possible cleansing operations are very diverse and highly dependent on the structure of the input data, they cannot be specified in the source configuration file. Instead, the user should provide a Python or R script that implements the data cleansing operations as a separate file. This script should respect some input/output constraints so that it can be ingested by inteGraph at runtime and incorporated into the ETL pipeline. After data cleansing is complete and cleansed data are staged, the pipeline moves on to schema mapping which involves converting data from the schema of the original data source to the schema of the knowledge graph, i.
For instance, to link the term 'Plant pathogen' found in the Fun Fun database to the corresponding class in the Soil Food Web Ontology, the component will retrieve all the classes whose label (or the label of one of its synonyms) matches exactly the lookup term. If a single eligible class is found, the term in the input data is annotated with this concept identifier, here SFWO:0000159 which is the identifier of the class 'plant pathogen' in SFWO. A third annotation component allows the user to provide a YAML file containing a dictionary with term: concept pairs. Semantic annotation components can be chained together Fig. 5. InteGraph implements a declarative approach to KG construction, giving the user control over the creation of data integration pipelines through simple configuration files.
to handle mismatched terms (see the source configuration file example in Fig. 5b ). Fig. 6c  shows the result of applying semantic annotation on our example datasets. Once the relevant data have been linked to the corresponding concepts in the target ontology, the final step of schema mapping is the conversion of the annotated dataset into an RDF graph (Fig. 6d ). Inte-Graph uses RDF Mapping Language (RML)  [36]  rules to transform tabular data into RDF triples. RML is a declarative language for expressing rules that map data in heterogeneous structures to the RDF data model. These rules describe the desired graph structure, that is how the data and schema layers of the graph should be connected to each other. The schema mapping rules should be provided by the user as part of the data source configuration.
However, writing RML mapping documents is beyond the reach of most non-expert users. To face this issue, inteGraph enables to specify mapping rules in spreadsheets that are automatically translated into RML documents using Mapeathor  [37]  (Fig. 7 ). This provides a more user-friendly manner to declare mapping rules in a language-independent way. Finally, inteGraph applies Morph-KGC  [38] , a modern RML processing engine with a focus on speed and scalability, to execute the RML mapping rules and generate the RDF graph. Morph-KGC uses the RML rules to determine how the annotated data should be transformed into RDF triples. The RML rules are applied to each row in the annotated data to generate the RDF representation of the information in the row. In case of missing data (e. g. a taxonomic entity that could not be mapped to the target taxonomic), the RDF triples that use the missing data are not materialized.
RML rules processing results in a RDF graph which is the sum of the sets of RDF triples generated for each row. Fig. 6d  shows an extract of the RDF graphs obtained by applying RML rules to our example datasets. RDF graphs are staged in N-quads format, a serialization format for RDF data that associates each triple with an optional context value at the fourth position. This context value takes the form of a graph label, indicating which RDF graph the triple belongs to. This graph label is used to keep track of the data provenance (original data source) after the different RDF graphs are merged into a single KG in the triplestore.
Data loading The third and final step of the ETL data integration process is to save the RDF graph generated during the data transformation stage in an external RDF database, i. e. a triplestore. The triplestore must be set up beforehand, either on the same machine running inteGraph or on a dedicated server. Different triplestore implementations may use different techniques to ingest RDF data. InteGraph provides connectors to the following triplestore solutions: RDFox, GraphDB, and Virtuoso. InteGraph also supports loading RDF data to a triplestore using SPARQL Update operations. In our example, the trophic KG is stored on an instance of GraphDB Free. The GraphDB connector provided by inte-Graph simply loads RDF data to the triplestore using an HTTP POST request. At the moment, inteGraph supports full load only.
A schedule interval can be assigned to each pipeline, which determines when and how often the pipeline is run. Alternatively, the user can manually trigger the execution of a pipeline in Airflow's graphical user interface. This interface also allows to visualize the pipelines generated by inte-Graph and monitor their execution (Fig. 8 ). Airflow can handle failures in ETL operations by retrying them a couple of times. If the error persists, the user can easily explore the logs of the failing task, identify the cause Fig. 6. An illustration of the application of data transformation to our example datasets (D1: Fun Fun , D2: Carabidae diet data from BETSI, D3: Collembola trophic interaction data from GloBI). Fig. 6d  shows the set of RDF triples (in Nquads format) generated from the first row of each data table.
e. the original source) of the triples, unless explicitly requested. This means that the set of RDF triples returned in response to a SPARQL query may contain facts originating from different data sources. The KG provides the user with a unified view of the original data sources through querying, and enables combining multisource information as part of a query response. Fig. 9  shows a SPARQL query searching the KG for phytophagous species. The same query returns both springtail and carabid species, whose dietary information originates from the GloBi and BETSI databases respectively. This simple example shows how a single query against the KG can retrieve information from multiple sources simultaneously, thus greatly facilitating integrative studies across taxonomic groups and/or trophic levels.
Making implicit knowledge explicit The semantic data integration process builds a KG by linking heterogeneous datasets at the schema level using an ontology. During the process, the data layer of the KG is populated with the factual information stated in the different datasets and transformed into knowledge through semantic annotation and transformation into a RDF graph. Based on these explicit facts, additional knowledge that is not explicitly present in the data can be derived using reasoning. This ability is a direct Fig. 7. InteGraph allows the user to specify schema mapping rules in a spreadsheet, which are automatically converted into RML rules using Mapeathor. The RML rules are executed using Morph-KGC to generate the RDF graph. Fig. 8. A high-level view of the ETL pipeline for the Fun Fun database in the Airflow user interface.
Fig. 9. Example of a SPARQL query returning the species names of phytophagous taxa. ?x denotes a variable called x. The LIMIT keyword is used to limit the number of results to the first 10 entries. The query returns information about both phytophagous springtails (from the GloBi database) and phytophagous carabid beetles (from the BETSI database). consequence of OWL (the standard language for specifying ontologies) being based on a subset of first-order logic. Therefore, automated reasoners can be employed to evaluate the logical implications of the knowledge encoded in the ontology on the explicitly stated data. There are two principle strategies for logical inference: forward chaining and backward chaining  [39]. Forward chaining, also known as materialization, derives all the facts that can be logically deduced from the existing facts and a set of logical rules, and stores these inferred facts in the triplestore for later querying.
Performance InteGraph relies on a number of external tools with a strong focus on scalability (e. g. GNparser for scientific name parsing, Morph-KGC for RML rules execution). This, combined with Airflow's ability to run independent tasks in parallel, makes inteGraph itself quite efficient at handling large datasets in a reasonable time. In our experiments, we were able to convert tabular data with over 440 K rows into an RDF graph in about 6 min on a laptop with twelve 2. 60 GHz Intel Core i7 CPUs and 16 GB of RAM. Currently, the main bottlenecks are taxonomic mapping, which in some cases may require many calls to web APIs, and logical inference, the performance of which depends on the types of reasoning and the optimisations implemented by the triplestore (inte-Graph provides no reasoning component, the inference is left entirely to the triplestore).
With the ability to chain semantic annotation components, including user-provided dictionary-like mapping files, inteGraph is able to convert most of the input data into RDF, with however some entries being dropped because they cannot be linked to concepts in the target ontology. Most of the time, this happens because the taxonomic entities could not be mapped to the target taxonomy. This can be due to the source and target taxonomies being incompatible, the taxon name being ambiguous or deprecated, etc. For example, in the FunFun database, 41 of the 508 unique taxa could not be mapped to the NCBITaxon ontology, resulting in 15% of the input data being dropped during the data integration process. In the Carabidae dataset extracted from BETSI, this proportion is only 0. 3% (with 18 of the 5491 unique taxa for which inteGraph could not find a correspondence in the NCBITaxon ontology).
Discussion Multitrophic studies require harmonizing and integrating datasets across a large variety of taxonomic groups and trophic levels. Despite considerable efforts to make more biodiversity data freely available in a (semi-)structured format, the multiple dimensions of data heterogeneity (semantic, structural, syntactic) constitute a major obstacle to the interoperability of data sources  [3]. Here, we introduced a practical approach to data integration that aims at making heterogeneous and distributed biodiversity data sources interoperable as part of a single KG. KGs provide a unified representation of disparate data sources and allow for retrieving data across these sources using a single query. By using ontologies as global schemas, they add semantics to the integrated data, making it easier for humans and computers to interpret the data and for reasoners to infer additional facts.
As seen from the example discussed in the Results section, the ability to reason about the data in our trophic KG opens avenues for automatic classification of soil organisms, which can facilitate the reconstruction of consistent soil food webs from multisource data. In addition, KGs provide support for a number of applications  [41] , including both in-KG, e. g. link prediction, error detection, and out-of-KG applications, e. g. relation extraction from text, recommender systems, etc. Despite their many advantages, KG construction is currently out-ofreach for most biodiversity data providers and consumers as they require in-depth expertise in Semantic Web technologies. InteGraph is an attempt to make semantic data integration and KG construction more accessible to the biodiversity science community.
Requiring little or no code and minimal knowledge of the Semantic Web, inteGraph facilitates the processes of converting a biodiversity dataset into a KG and of integrating multiple datasets into a single KG. Given a set of distributed data sources and a target ontology, inteGraph allows the user to control the creation and execution of reproducible ontology-based data integration pipelines through a set of simple configuration files. This Fig. 10. Example of a SPARQL query returning the trophic groups to which Entomobrya ligata belongs. A trophic group is defined in the Soil Food Web Ontology as 'a collection of organisms that feed on the same food sources and have the same consumers'  [7, 31]. SFWO provides a logical formalization of the hierarchical classification of soil consumers proposed in Ref.
[40]. declarative approach relieves the user of the implementation burden. Instead, the user can focus on the desired structure of the target KG and on the schema mapping rules needed to transform the input data into RDF graphs. InteGraph relies on high-performance third-party tools (gnparser, nomer, Morph-KGC, Airflow), which guarantees a certain ability to scale to large datasets. The viability of our approach has been tested by creating a KG of soil trophic ecology from multiple open trait databases, using the Soil Food Web Ontology and the NCBITaxon ontology as the KG schema. Currently, inteGraph is at the proof-ofconcept stage. It still needs some development to make it more robust, scalable and user-friendly. We also plan to add more advanced features in the future, especially regarding data provenance tracking and continuous KG updating.
Although it represents a significant advance in the field of ontologybased biodiversity data integration, inteGraph suffers limitations related to current practices in biodiversity data management. First, inteGraph requires the data sources to provide data in a (semi-)structured format through a programmatic interface, e. g. a URL to download the data file or a web API that handles HTTP requests. Still lots of data about soil biodiversity are not accessible this way, e. g. data from the BETSI database must be downloaded manually. We are confident that this situation will become less frequent in the future. Second, as a top-down approach to KG creation, inteGraph requires a predefined ontology to act as the mediating schema to link heterogeneous data sources. Creating an ontology to model knowledge in a domain of interest is a complex process that requires a significant investment of time and effort.
Ontology engineering asks for a group of experts to produce a consensual conceptualization of the domain. For instance, in the domain of soil trophic ecology, this means trying to harmonize the use of diet terms that may have different meanings from one taxonomic group to another. However, we believe that the result is worth the effort, as a properly designed ontology can benefit the whole community by facilitating knowledge sharing, dataset standardization and, ultimately, data integration. Finally, although it aims to make semantic data integration more accessible to a non-expert audience, inteGraph still requires a minimum of knowledge of Semantic Web technologies (RDF data, ontologies, SPARQL queries …). Just as the environmental community has begun to embrace new artificial intelligence tools from recent developments in deep machine learning, we encourage the community to take an interest in Semantic Web tools for better biodiversity knowledge management.