dataesr/scientific-paragraphs-categorization huggingface dataset:
Desired mathematical formula features F1. Enable FAIR mathematical formulas. Representations in machine-readable formats such as RDF or XML are more FAIR than programming language scripts. Besides, RDF is the preferred format for the Linked Open Data (LOD), which is why RDF is a better choice. The FAIR  [1]  guideline proposes good practices for data management. This guideline highlights the importance of applying FAIR principles for algorithms, tools, workflows and analytical pipelines. Additionally, a representation close to the mathematical notation may promote to share the mathematical formulas for a wider community including non-IT experts and can be aligned with the mathematical knowledge representation effort  [13]. F2. Include the computation-metadata. The metadata facilitates the human comprehension and the traceability.
F6. Enable nested ANRs. Nested ANRs rely on the resulting triples of other ANRs. This feature is required to represent more complex mathematical formulas in a modular manner and can allow handling heterogeneous data. For instance, let us consider an ANR over cities of DBpedia, a well-known KG in the SW that uses as input the areaTotalKm (total area in squared kilometres) property. Some cities in the data use this property to express the area of a city while some others have this information in square miles (areaTotalSqMi). Therefore, the property areaTotalKm can be derived by performing another ANR transforming  - - ✓ ✓ - - - ✓ ✓ 4 LDScripts [21] - - ✓ ✓ / / - ✓ ✓ 5 SPARQL𝜆 [22] - - ✓ ✓ ✓ / - ✓ ✓ 5. 5 SPIN [9] - ✓ ✓ ✓ - / - ✓ ✓ 5. 5 Ontological-mediated computation Ontologies [23-25] - ✓ ✓ ✓ - - - - - 3 RDF-OpenMath [8] ✓ ✓ ✓ - - - ✓ - - 4 Ontological-reasoning-based computation Iannone & Rector [26] - - - - - - - - - 0 Bischof & Polleres [12] - - - - ✓ ✓ ✓ ✓ - 4 Rule-based computation SWRL [10] / - - ✓ - / ✓ ✓ - 4 SHACL [3] SPARQL rules - - ✓ ✓ - / - ✓ ✓ 4.
5 SHACL [3] Triple rules ✓ / - ✓ - / / ✓ - 4. 5 SHACL [3] Value rules ✓ / - ✓ - / ✓ ✓ - 5 Bischof et al. [11] - ✓ ✓ / - / ✓ - / 4. 5 the miles to kilometres. If this feature is not available, cities containing the property areaTotalSqMi will be omitted. F7. Link the ANRs to the KG data to facilitate query-answering. The mathematical formulas should be associated to the existing KG data in order to create an abstraction of derivable data. Thus, a user can query this data even if it is not yet calculated. Approaches that produce arbitrary triples are not desired since they require materialisation for the query-answering task. A second alternative is to navigate and interpret complex expressions to identify the derivable triples, which could complicate the query-answering task. A last alternative is to require the user to manually add in the query the ANR URIs, which is not desired since users should not be aware about these details.
In that sense, to know in advance the derivable properties facilitates the query-answering in the presence of ANRs. F8. Use SW engines to perform ANRs. It facilitates the interaction with RDF graphs and the system adoption. It avoids the installation of third-party software. F9. Support more complex outputs. Similarly to the mathematical formula inputs (F4), the related approaches often allow to produce only data properties with numerical literals. It is desired to produce more complex models such as quantity values. In that regards, ANRs should be extended with the expressivity of existential rules  [19]. Regarding the desired features, we propose the Table  1  to compare the related approaches. We use the ''✓'' to state that approaches meet the feature and the ''/'' for cases where approaches can be easily adapted to meet the feature.
Query-language-based computation SPARQL1. 1  [20]  is a W3C standard to query RDF datasets. SPARQL queries can represent aspects of ANRs. Within queries it is possible to express algebraics and aggregations (meet F3). These queries allow to navigate quantities and complex data models (meet F4). However, queries are not expressed in a LOD format (e. g. , RDF); and they are far from the mathematical notation (not meet F1). Also, the structure of a query is not extensible to add the computation-metadata (not meet F2). Concerning the rule aspects, an ANR can be represented as a CON-STRUCT query, which enables to produce new derived graphs (meet F9). Additionally, SPARQL queries facilitate the interaction with RDF datasets and they are performed by SW engines (meet F8). However, SPARQL does not enable query-answering on the derived data since one query cannot trigger other stored queries, and we are assuming that ANRs are stored as CONSTRUCT queries (not meet F7).
For the same reason, ANRs in SPARQL cannot refer to each other (not meet F6). An alternative to enable query-answering using SPARQL would be to materialise the derived data in advance obtaining a new enriched graph which nevertheless is not a desired feature (not meet F5). LDScript  [21]  is a SPARQL extension that allows to express functions directly in SPARQL. LDScript contributes to the modularity issue of SPARQL: a function can refer to other functions and a query can use them (partially meet F5 and F6). However, these functions are very restricted, they only allow to store a procedure, but they cannot be linked with the KG data to enrich the data (not meet F7). SPARQL𝜆  [22]  extends the capabilities of SPARQL1. 1 with a functional programming approach. SPARQL𝜆 enables to define data analysis pipelines by injecting partial graphs calculated on the fly.
This approach allows to store templates of RDF producing functions. Later, in a query, users can refer to the graph derivable from the template (meet F5). SPARQL𝜆 templates can trigger other templates enabling nested ANRs (meet F6). However, these templates do not enrich the data, users defining a query must know the template URI to obtain the derived data, and cannot query it directly (not meet F7). On the other hand, the SPARQL Inference Notation (SPIN)  [9]  is an RDF vocabulary to store SPARQL queries in RDF. In RDF it is easier to add metadata (meet F2). SPIN can define ANRs as CONSTRUCT queries that are stored in RDF. Although SPIN allows to store SPARQL queries in a LOD format, this representation is far from the mathematical notations and it may be difficult for users not familiar with SPARQL to interpret the ANRs (partially meet F1).
Ontological-mediated computation An ontology is an explicit, formal specification of a shared conceptualisation  [27]. Ontologies allow to add the computation-metadata with explicit semantics. A delegated engine can use later these metadata to perform specific tasks (e. g. to trigger scripts). Proposals like the Function Ontology (FO)  [24] , the ontology of functions in R  [25]  and the ontology for quantitative research  [23]  enable a detailed definition of the computation-metadata (meet F2). However, they neglect the mathematical expressions (not meet F1), which are not explicitly declared. Often these mathematical expressions are defined in a target programming language (e. g. R, Python, Java). Also, these approaches do not intent to enrich the KG data (not meet F4, F7 and F9). RDF-OpenMath  [8]  is an effort to represent mathematical objects in RDF, based on OpenMath: an standard to exchange mathematical objects in Computer Algebra Systems (CAS).
In RDF-OpenMath the mathematical expressions are explicit (meet F1 and F2) and accompanied by a concrete semantic of symbols. 4  Nevertheless, expressions are evaluated by Computer Algebra System (CAS) (not meet F8). At the moment, none of these systems is a W3C standard or recommendation. The current version of RDF-OpenMath does not allow inputs and outputs different from literals (not meet F4 nor F9). Although RDF-OpenMath allows to link the mathematical formula to the KG data (meet F7), the authors do not report any procedure to perform the formulas on demand (not meet F5). Besides, it is not clear if they allow nested ANRs (not meet F6).
Rule-based computation Rule languages allow to define mathematical formulas through built-in functions. The Semantic Web Rule Language (SWRL)  [10]  is a language to define rules tightly aligned with OWL axioms. However, concerning the desired features, a SWRL rule cannot produce new instances (not meet F9) and lacks support for more advanced calculations (not meet F3) like aggregations  [30]. SWRL rules also neglect the computation-metadata and they never were sufficiently adopted to become a W3C recommendation. Moreover, while SWRL rules can be expressed in RDF, the mathematical formulas cannot be stored as expressions in a linked data format (partially meet F1). Since SWRL rules have native ontological reasoning, KG data can be linked with the mathematical formulas (meet F7). For the query-answering problem (virtual ANRs), SWRL rules can lead to rounding and infinite loop errors  [12].
This is because SWRL rules are not intended to handle numerical data. Examples of rule languages like Apache Jena 5 rules and N3  [31]  present similar limitations (see Table  2 ). The Shapes Constraint Language (SHACL)  [3]  is a W3C recommendation since 2017 for RDF graph validation. While SWRL rules extend the expressivity of OWL, the SHACL semantics is rather given as mappings to SPARQL endpoints  [32]. SHACL improves SPIN by focusing more in the validation and representing the constraints as RDF 5 https://jena. apache. org/. triples. With SHACL, one can express constraints such as ''a person only can have one property value for their weight and height '', ''weight and height should be decimal values''. Besides, SHACL is gaining more popularity within the Semantic Web community. The machinery used to validate SHACL constraints can be adapted to perform other useful tasks in KGs (e.
g. , performing functions and rules). The SHACL Advanced Features (AF) draft  6  is an example of those extensions. In this draft, the authors present three kinds of rules that differ in their expressivity and their representation: • SPARQLRules (Listing 3) are rules that contain a SPARQL CON-TRUCT query. These rules allow to produce an arbitrary graph for each SHACL focus node (meet F9). However, to enable queryanswering, these rules require materialisation (not meet F5). Note as well that linking an arbitrary graph to the KG data is complex without materialisation (not meet F7). Besides, in these kind of rules the mathematical formulas, in form of queries, are added as text (not meet F1 nor F2). Notice that all the SHACL rules require a manual definition of the execution order (partially meet F6).
Experimental evaluation Two experiments were conducted to evaluate the performance of QAVAN. The first experiment aimed (i) to compare QAVAN against a materialised approach (considered as a baseline), and (ii) demonstrate the benefits of QAVAN in terms of processing time and disk space. This experiment used the Météo-France dataset  [36] , a typical scientific dataset containing meteorological records. The second experiment aimed to show the generality of our approach for representing real mathematical formulas using two public heterogeneous datasets: DBpedia  [37]  and Eurostat. 9  Based on both datasets, we created a dataset containing instances of the class Settlement and its subclasses City and Town. Additionally, this experiment allowed to show the interest of the extensions of QAVAN (See Section 4.
Implementation and settings QAVAN was implemented based on Apache Jena 10  and SHACL APIs. 11  The QAVAN approach along with the datasets used in the experimental evaluation and its results are available at the git repository: https://github. com/felipe-vargas-inrae/qavan as well as the PerfectRe-fEQ implementation https://github. com/felipe-vargas-inrae/PerfectRef EQ The experiments were conducted in a machine with the following features: Ubuntu 20. 04. 4 LTS, Intel(R) Core(TM) i7-8665U CPU @ 1. 90 GHz Cores: 6. Additionally, we used as JVM parameters: -Xms30 g -Xmx30 g -Xdump:none -XX:+UseParallelGC to control the memory usage and the Java Garbage Collector.
Data graph The Météo-France 12  dataset  [15]  contains time-series data over 500 weather stations in a delimited area on the Brittany coasts. This dataset contains instances of weather observations taken in a specific time period (initially described in a CSV format and transformed into RDF for the sake of this experiment). In particular, each weather observation is described by six properties: :windSpeed, :airTemperature, :air-RelativeHumidity, :precipitation, :windDirection and :dewPointTemperature 5. Note that we choose to model all these properties as object properties storing not only the values but also their units of measurements. Fig. 1  shows our data model which uses terms from QUDT  [18]  and SOSA  [38]  ontologies. These ontologies are widely used in numerous research domain to model observational data.
To evaluate the robustness of our approach against different sizes of KGs, we splited the Météo-France dataset in five datasets varying in the number of weather samples as shown in Table  6. For each dataset, Table  6  includes the number of triples and the RDF file size. The name of each dataset 𝐷 ⟨𝑁⟩ refers to the top 𝑁 weather stations. For instance, D1 refers to data coming from one weather station whereas D20 to data coming from 20 stations. Therefore, the bigger the N the bigger the size of the dataset evaluated in this experiment.
Shapes and ANRs To validate our approach over the five datasets created from Meteo-France dataset, we generate different sets of ANRs (e. g. 5, 10, 20,. , 100) using a random generator. More precisely, the ANRs are generated using the DASH operators (i. e. , add, subtract, divide, multiply), the data graph properties (e. g. :windSpeed, :airTemperature, :relativeHumidity) and random numerical literals.
Evaluation queries In order to demonstrate the behaviour of our algorithm QAVAN against different types of queries, we defined eleven queries summarised in Table  7  covering different cases of time frames and calculable properties. Since the datasets contain time series data, we employ time as filter (e. g. observations of one-day) to assess QAVAN with different query result sizes. Note that all and 50% in the derivable properties is relative to the number of ANRs whereas one is always fixed. For instances, 50% of 100 ANRs represent a query asking for 50 calculable properties. This evaluation aims to answer the following research questions: (RQ1) Is QAVAN able to improve the processing time for queries requiring few instances in their answers? (RQ2) Is QAVAN able to improve the processing time for queries involving few properties to be computed? (RQ3) Which processes are more time-consuming for QAVAN? (RQ4) How many ANRs do materialisation unfeasible considering different data graph sizes?.
Results We assess the performance of QAVAN on different datasets and queries by using as a baseline the query 𝑞 0 that retrieves all the ANRs over all the instances (see Table  7 ). Since 𝑞 0 is computing all the ANRs available for all the data, it is employed to simulate a materialised approach (see Algorithm 1) (see Table  8 ). Results of queries processing time: Fig. 2  shows the impact of the number of ANRs on the processing time over 4 different queries, 𝑞 0 , 𝑞 3 , 𝑞 6 and 𝑞 9. All these queries contain the full list of calculable properties from the set of ANRs available. Note that 𝑞 0 retrieves data concerning a full year, 𝑞 3 data of a specific day, 𝑞 6 data of a specific week and finally 𝑞 9 data of a specific month. Fig. 2  aims to show the boundaries of the materialised approach and how QAVAN improves the processing time for queries that involve less instances.
We observe that the materialised approach (𝑞 0 ) processes fast (less than 1 min) small datasets (e. g. , D1)  over few ANRs (e. g. , 20). This is not the case for big datasets (e. g. , D50) over few ANRs (e. g. , 20) that demonstrate an excessive processing time (about two hours). Note that in some domains a two-hour delay may be inadequate. Even worst was the case of the big dataset D50 with a large number of ANRs (e. g. , 100) where no answer was obtained due to memory allocation issues. Indeed, this behaviour was observed from 30 ANRs. Therefore, query-dependent approaches such as QAVAN may be the only alternative in cases where materialisation is infeasible (e. g. , D50 and 40 ANRs). Results on Fig. 2  shows that QAVAN when evaluating queries 𝑞 3 and 𝑞 6 is able to provide results for all dataset scenarios avoiding the memory allocation issues, and also improving the processing time.
Moreover, on the scenario of evaluating the query 𝑞 9 against the dataset D20 with 100 ANRs available, QAVAN demonstrates a great improvement in the processing time over the materialised approach (𝑞 0 ), i. e. , 8 min against 122 min showing an improvement of 92%. Thus is explained due to the capability of QAVAN to avoid unnecessary computations when less instances are required in the query results While our experiments demonstrate the superiority of QAVAN against the materialised approach in terms of processing time, the size of datasets has an impact on QAVAN as well. Therefore, the evaluation of 𝑞 9 (returning 3,707,26 instances) against the biggest dataset (D50) showed that QAVAN experimented a higher processing time going up to 23 min whereas for 𝑞 6 and 𝑞 3 it was about 9 and 7 min, respectively.
Note that queries returning numerous instances can have similar time executions for both approaches since no optimisation strategy can be applied for the case of QAVAN. Therefore, we can say that QAVAN is a much more appropriate for queries returning few instances against big datasets over many ANRs in terms of efficiency and processing time. Fig. 3  shows the processing time for all the evaluated queries, grouped by the time filter. Since these results are more comparable, figures are not presented in logarithmic scale. Results depicted on Fig. 3  aim to demonstrate that QAVAN improves the processing time when less calculable properties are involved in the queries. In particular, Fig. 3 (a) compares queries that retrieve all the instances (one-year queries) and vary the number of calculable properties.
These results show that QAVAN improves the processing time when less calculable properties are invoked even against big datasets (e. g. D50 and D20). In particular, the evaluation of 𝑞 1 against the biggest dataset (D50) is processed in less than 4 min even for a large number of ANRs (around 100). Comparing 𝑞 1 and 𝑞 2 we observe an increase in the processing time since the second query expects more calculable properties in its answer. Note that, when 𝑞 2 is executed on D50 with 50 ANRs it reaches its limits. In Fig. 3(d) , we observe that for one-month queries (returning 370,726 instances), the query 𝑞 11 that contains 50% of the calculable properties exhibits a processing time improvement of 60% compared to 𝑞 9 (i. e. , 23 min for 𝑞 9 and 14 min for 𝑞 11 ), when evaluating against the dataset D50 over 100 ANRs.
To conclude, in this experimental evaluation we notice that QAVAN is able to handle all datasets involving millions of triples as long as the queries contain up to 50% of the calculable properties found in the ANRs. Additionally, when queries contain few calculable properties, QAVAN performs well even against datasets containing millions of triples and hundreds of ANRs. On the other hand, when queries return few instances, the number of calculable properties does not have a relevant impact on the processing time of our approach. Effect of query steps on processing time: Fig. 4  exhibits the steps that are taking place in QAVAN, (e. g. load a graph G, perform ANRs, etc, using different colours) in order to demonstrate the importance of each step in the processing time for queries 𝑞 0 , 𝑞 3 , 𝑞 6 and 𝑞 9.
Our main observation is that concerning the same query, each time step takes approximately the same proportion of time independently of the dataset size. In comparison, Fig. 4 (a) shows the materialised approach (𝑞 0 ). We notice that in this scenario, the step 'Perform E' is the most time-consuming step and dominates the total time, in particular when the number of ANRs increases. Conversely, 'Load G' is more time consuming when few ANRs are invoked. However, for 100 ANRs 'Load G' is irrelevant in terms of execution time as most of it is allocated to 'Perform E'. 'Store Rs' also gains relevance when the number of ANRs increase, even overcoming the 'Load G' process. Fig. 4 (d) details the evaluation of the one-month query 𝑞 9. Since this query returns less instances than 𝑞 0 , for few ANRs the 'Load G' step is more dominant, whereas for many ANRs 'Perform E' takes more time.
Moreover, for queries returning few instances as those shown in Figs. 4(c ) and 4(b), 'Load G' step dominates the entire execution. These results demonstrate that 'Load G' and 'Perform E' are the most time-consuming steps during the execution of the queries. Thus, efforts may be focused in optimising those steps in the future. Inference graph size results: Fig. 5(a)  shows the storage saving of QAVAN during the evaluation of several queries and against the materialised approach. To guarantee a uniform comparison, we define a ratio relative to the evaluation of 𝑞 0. More precisely, we present the size of the new graph produced by a query evaluation divided by the size of the corresponding graph produced by 𝑞 0. Moreover, for a further analysis, we demonstrate on each sub-figure the size of the initial dataset 𝐺.
Note that 𝐺 corresponds on Fig. 5 (a) to D10, D20 or D50. The comparison of 𝐺 and the inference graph of 𝑞 0 (𝐺 ′ 𝑞 0 ), for all the datasets, shows that after 20 ANRs 𝐺 ′ 𝑞 0 becomes bigger than the initial graph 𝐺. Bear in mind that an inference graph bigger than an initial graph could be undesired. We notice that D10 and D20 display similar results. This is not the case for D50 that experiments memory allocation issues to evaluate 𝑞 0 from 20 ANRs. Concerning the other queries referring to QAVAN, we observe that evaluations improve the 𝐺 ′ size reducing its size. For instance, evaluation of 𝑞 6 and 𝑞 3 require a 𝐺 ′ 90% and 99% smaller than 𝐺 ′ 𝑞 0 , respectively. 5(b) exhibits similar results depicting the storage saving for small number of calculable properties. Size of 𝐺 ′ for 𝑞 10 and 𝑞 11 were smaller than the one for 𝑞 9.
We also noticed that for the materialised approach with few ANRs 'Load G' was more time-consuming whereas for many ANRs 'Perform E' overcomes 'Load G'. Answer to RQ4. We found a boundary for the materialised approach on 30 ANRs against a big dataset (106,506,111 triples and 4,346,937 instances) where this approach does not provide any results since its execution never finishes. On the contrary, QAVAN succeeded to provide answers to such contexts.
Evaluation on DBpedia and eurostat datasets We evaluated our approach's behaviour on from two very well known open datasets, DBpedia  13  and Eurostat Urban Audit. 14  Note that similar data coming from DBpedia and Eurostat have been also used in the Bischof et al. [12]  experiments. This experimental evaluation allowed us to demonstrate the utility of our approach in datasets coming not only from experimental data but also in other domains. Along with this goal we also exhibit the importance of the inference and nested ANRs features of our approach in order to obtain more valuable answers. DBpedia is currently one of the biggest and more used public KGs. DBpedia is an effort to structure and publish Wikipedia 15  data on the web. On the other hand, Eurostat is a portal of the European Union (EU) that periodically publishes statistics and indicators concerning its country members.