# Ontology evolution from resource description framework streams using possibilistic axiom scoring

Alda Canito, Jérôme David, Juan Corchado, Goreti Marreiros

► To cite this version:

Alda Canito, Jérôme David, Juan Corchado, Goreti Marreiros. Ontology evolution from resource description framework streams using possibilistic axiom scoring. Semantic Web – Interoperability, Usability, Applicability, 2025, 17 (1), pp.1-33. 10.1177/22104968251406727. hal-05463709

HAL Id: hal-05463709

https://hal.science/hal-05463709v1

Submitted on 17 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Ontology Evolution from Resource Description Framework Streams Using Possibilistic Axiom Scoring

Semantic Web
Vol. 17(1) 1-33
© The Author(s) 2025
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/22104968251406727
journals.sagepub.com/home/swj

<figure><img src="image_2.png" /></figure>

Alda Canito<fnref n="1" /> ID, Jérôme David<fnref n="2" />, Juan Corchado<fnref n="3" /> and Goreti Marreiros<fnref n="1" />

## Abstract

Evolving an ontology involves re-learning, re-enriching and re-validating knowledge in the face of changes to the domain, and techniques applied for them can be adapted to ontology evolution. The possibilistic approach to axiom scoring has been applied over complete and large datasets in ontology learning. This paper presents an adaptation of the possibilistic approach to axiom scoring to the context of Resource Description Framework data streams for ontology evolution, a scenario which forcefully deals with incomplete and time-dependent data. Possibilistic axiom scoring is used in two distinct scenarios: (1) With previously known property axioms, allowing for the exploration of the effectiveness of the approach in a scenario in which no incorrect data was present; and (2) in an evolving knowledge scenario, in which neither the properties nor the axioms were known and the dataset was obtained from publicly available sources, possibly both incomplete and with errors. Results show the effectiveness of the approach in accepting/rejecting axioms for the ontology's properties. The different approaches to possibility and necessity proposed in literature are recontextualized in terms of their bias towards selective confirmations or counterexamples - showing that some axioms benefit from a more lenient approach, while others present a lower risk of introducing inconsistencies by having harsher acceptance conditions.

## Keywords

ontology evolution, time-Sensitive data, data streams, property axioms

Received: May 30, 2024; accepted: August 11, 2025

Editor: Aidan Hogan, Associate Professor in the Department of Computer Science (DCC) at the Universidad de Chile and Associate Researcher of the Millennium Institute for Foundational Research on Data (IMFD), Santiago, Chile; Solicited reviews: Andrea Tettamanzi, Professor at Université Côte d'Azur, Nice, France; Stephan Mennicke, Postdoctoral Researcher of Knowledge-Based Systems, Dresden University of Technology, Dresden, Germany.

## I Introduction

Ontology Evolution – especially when performed automatically or semi-automatically – cannot be detached from other subfields of ontology studies in computer science, such as ontology learning, enrichment, and validation (Khadir et al., 2021; Polleres et al., 2023). Many steps and techniques are transversal to these fields; consider, for example, how evolutionary processes occur: New knowledge needs to be learned so it can be formalized into the ontology, and data can be used to further enrich the evolving schema into a more expressive and precise result. Ontology evolution is, in a way, the

## Corresponding Author:

Alda Canito, GECAD/LASI, ISEP, Polytechnic of Porto, rua Dr. António Bernardino de Almeida, 4249-015 Porto, Portugal.

Email: alrfc@isep.ipp.pt

<footnote n="1">¹GECAD/LASI, ISEP, Polytechnic of Porto, Porto, Portugal</footnote>

<footnote n="2">²Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LIG, Grenoble, France</footnote>

<footnote n="3">³Department of Computer Science, University of Salamanca, Salamanca, Spain</footnote>
process of re-learning, re-enriching and re-validating an ontology in the face of changes to the domain, particularly when
these are triggered by the data itself.

While many ontology learning approaches imply the acquisition of data through text, more recently there has been a shift towards using continuous streams of (structured and semi-structured) data (Khadir et al., 2021; Polleres et al., 2023; Su et al., 2016). Data streams carry, implicitly or explicitly, a time dimension that can and must be taken into consideration if it is meant to guide evolutionary processes (Esling & Agon, 2012; Hamilton, 1994; Polleres et al., 2023; Su et al., 2016). This means the ontology is not learned once, but that learning and evolving the ontology are inextricably linked and the data used to trigger those processes is both limited and transient.

Ontology learning and evolution solutions tend to focus on the identification and materialization of changes in concepts and roles and the hierarchies between them, but the same attention has not been given to the axiomatization of said structures (Canito et al., 2022) – particularly when dealing with the inherent incompleteness that comes with data obtained via streams.

The Time Constrained instance-guided Ontology Evolution (TICO) (Canito et al., 2023) tool is an ontology evolution framework that analyses new ontology individuals to understand if the concepts defined on the ontology have changed over time. The framework implements a set of operators that compare definitions present in an ontology to the structure of individuals incoming through a Resource Description Framework (RDF) stream. The result of the application of these operators is a set of evolutionary actions that can be triggered to produce changes in the ontology – generating new versions of specific concepts and properties to suit the patterns found in the individuals. If sufficient difference between the individuals of the concepts being analyzed and the version of them asserted in the ontology is identified, TICO uses a 4-D Fluents (Burek et al., 2019) approach to reify new, disjoint definitions of the concept for each time period – or Time Slice – in a strictly positive monotonic fashion; the tool aims to iteratively evolve an ontology as a result of the analysis of small numbers of individuals. The architecture of TICO has been described in Canito et al. (2023) and allows for the analysis of streams of RDF individuals to extract potential evolutionary actions that add temporally-bound axioms to the ontology. In this paper, the authors use the architecture of TICO as a base for stream-guided ontology evolution with a focus on the identification of ontology property constructors. This is done by analysing extensional evidence for and against each of the ontology roles’ constructors and ascertain if the data shows enough support for their inclusion in newer versions of the ontology.

The work presented in this paper aims to assess the degree to which it is possible to identify changes in OWL property axioms through the analysis of incomplete, unbounded and changing RDF data provided by streams, and to establish which metrics and assumptions/constraints are more suitable for this task. To do so, axiom testing analysis from both statistical and possibilistic perspectives will be executed. Additionally, when applying the solution to an ontology evolution scenario, this work aims to assess if and how the knowledge already present in the ontology should affect the decision to include/exclude suggested axioms.

The main contributions of this work are therefore:

(1) Adaptation of the possibilistic approach to axiom testing as described in the works of Tettamanzi et al. (2017) to the context of streams of RDF individuals/instances, followed by an extensive, in-depth analysis of the robustness of the proposed metrics. This includes the analysis of the effects of the number and variety of the individuals that can be analysed simultaneously when searching for potential axioms in RDF data streams.

(2) Combining acceptance/rejection index (ARI) with other metrics in order to accommodate for the potential existence of errors in data: The percentage of selective confirmations w.r.t the support and an evolving form of the ARI which is informed by the knowledge present in previous versions of the ontology. For this purpose, the approaches will be tested against an ontology generated from publicly available data, for which no *a priori* information about property characteristics is known.

The rest of the paper is organized as follows: Section 2, Background, which contextualizes the work in the field of ontology learning/evolution and describes existing works on axiom scoring; Section 3, Property Axiom Scoring, which describes the solution and details the definitions of the axioms pertaining to each of the property characteristics in OWL and how to evaluate their presence in an RDF data stream. Section 4, Effects of Sliding Window size and suitability of ARI for axiom scoring, establishes the suitability of the possibilistic approach and the ARI to axiom scoring in RDF streams and compares its performance with that of traditional information-retrieval metrics; Section 5, titled Accommodating for errors in data and the effects of previous knowledge in axiom-inclusion decisions, compares a combination of ARI and selective confirmations with an evolving form of ARI to assess how they deal with potentially incomplete and noisy data obtained from public datasets. Finally, Section 6 presents the Conclusions.
## 2 Background

Changes to domain knowledge must first be identified and quantified, so that they can be materialized into executable actions that will modify an ontology into a new version of itself – i.e., evolutionary actions (Kondylakis & Papadakis, 2018; Peixoto et al., 2015). To identify if there is a need for a change at the ontology level – if and which evolutionary actions should be considered – it is often necessary to analyze data that is external to the ontology (Benomrane et al., 2016b; Cardoso et al., 2018). Examples of this include identifying changes in the way end users apply the concepts in the ontology to the data, (Kondylakis & Papadakis, 2018) evaluating corpora regarding the domain to extract new concepts and compare them to existing ones, (Benomrane et al., 2016a; Cano-Basave et al., 2016; Cardoso et al., 2018) frequently through the application of Natural Language Processing algorithms, and the analysis of datasets of structured data (Kondylakis & Papadakis, 2018; Peixoto et al., 2015) (such as RDF datasets). It is possible to conceive ontology evolution as ontology learning with extra steps: New concepts and roles must be derived from existing data, but they also must be compared and made consistent with previously established ones, or otherwise have to update their definitions.

The identification and execution of evolutionary actions when they concern the addition of classes and properties – and the hierarchies between them – are relatively well documented in literature (Canito et al., 2022; Ghidalia et al., 2024; Kondylakis & Papadakis, 2018). The identification of the axioms that could enrich them (e.g., class expression restrictions and property axioms), however, is not as popular – potentially stemming from the fact that many ontologies are often used as taxonomies and lack axiomatic complexity (Ghidalia et al., 2024). This axiomatic complexity, however, is particularly relevant for lower-level ontologies, which need to describe the intricacies of their domains with varying degrees of detail and can be used to determine the consistency of data they describe, and to derive implicit information using reasoners (Nguyen, 2021). For that, it is necessary not only to identify changes in concepts and roles and their relative novelty, but also to enrich them with axioms: Ontology evolution is the more useful the more it applies the precepts of ontology learning. Furthermore, (Canito et al., 2022; Ghidalia et al., 2024) detail the relative popularity of different types of evolutionary actions as they are described in literature, noting that while identifying new properties and property hierarchies is often considered, identifying and changing their property axioms in particular is still a relatively understudied field.

Ontology learning can be defined as the processes and techniques applied to design ontologies either automatically or semi-automatically (Khadir et al., 2021). According to Al-Aswadi et al. (2020) said techniques can be classified into two main categories: (1) Linguistic-based approaches and (2) machine learning-based approaches, which can be further divided into statistics or logic-based.

Linguistic-based approaches focus on the analysis of large corpora of text to identify potential concepts and the relationships between them (Al-Aswadi et al., 2020; Belhoucine & Mourchid, 2018; Venu et al., 2017). To do so, they seek for patterns and syntactic information in the text – making them particularly language-dependent. Machine learning-based approaches, on the other hand, can use different types of input data for their training: Both structured and unstructured. Statistics-based approaches are usually applied to identify the co-occurrence of terms, association rules and hierarchies (Al-Aswadi et al., 2020; Ghidalia et al., 2024). Logic-based approaches, on which the work described in this paper is grounded, use logical inference or inductive logic programming to derive rules from positive and negative examples found in structured datasets. This approach is particularly suited for learning rules and formalizing axioms. On the Ontology Learning Layer Cake (Buitelaar et al., 2005), which is used to describe the layers of the learning process and, by extension, the possible tasks it encompasses, rule and axiom extraction is depicted as the final sub-task in the process and the least explored in literature (Al-Aswadi et al., 2020).

In OWL, an axiom is a statement that expresses what is true in the domain described through the ontology. The number and type of axioms directly affect the expressivity of the ontology by adding more information to the description of classes, properties, and assertions, among others, and the relationships between them. For example, an ontology that can specify if a certain property is a mandatory element of a class, or how many times that role can be applied to an individual, is more expressive (and potentially more complete) than one in which those assertions are not established (Khadir et al., 2021).

Axiom testing is the process of evaluating the credibility of a given hypothesis concerning the relationships in a domain – the property axiom – by assessing whether the individuals of said domain (e.g., facts of an RDF dataset) confirm or deny a hypothesis (Tettamanzi et al., 2017) – i.e., whether they are confirmations or counterexamples of the axiom. The selective confirmation (Scheffler & Goodman, 1972) principle can be put into effect as well: A fact selectively confirms a hypothesis when not only it favours that hypothesis but also fails to confirm its negation. Not all facts are equally relevant for axiom testing, and, as such, the number of examples needs to be considered not in the context of all analysed instances, but only of those that do entail the relationships under scrutiny (Tettamanzi et al., 2017).
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Familial relationships between individuals in a stream.</figcaption></figure>

Property characteristics, from a perspective of axiom suggestion for ontology enrichment purposes, have been described in Bühmann & Lehmann (2012). This work describes enrichment methods that have been implemented as part of the DL-Learner framework. The approach involves using different queries to look specifically for axiom support in a triple store, considering both the count of examples and the average of the 95% confidence interval when suggesting axioms to the user. However, while the approach tackles the discovery and materialization of ontology axioms, including property axioms, it depends on (large) knowledge bases containing a complete collection of samples that can be analysed as a whole – and by analysing only these samples to generate possibilities it is, in a way, working under a closed world assumption. Similarly Sais et al. (2017) describe how to identify and materialize changes in property axioms but requires large and self-contained datasets to do so, falling under the same assumption.

The possibilistic approach has been applied in the works of Tettamanzi et al. (Malchiodi & Tettamanzi, 2018; Nguyen & Tettamanzi, 2019; Tettamanzi et al., 2014, 2015, 2017) for axiom testing against RDF facts in ontology learning and validation/evaluation scenarios. As the name indicates, the possibilistic approach deals with the degree of possibility of an event, such as an axiom – which falls in a range between impossible and possible [0,1]. Possible, unlike probable (from probability distributions), does not mean that the axiom must be true: Only that it is compatible with the known state of the world. While probability theory is suited for the representation of random and observed phenomena, the possibilistic theory better reflects how to deal with incomplete knowledge. In Tettamanzi et al. (2014) (and, by extension Tettamanzi et al., 2017), the authors detail how using such an approach is more suitable for candidate axiom scoring than statistical/frequentist approaches, claiming that the nature of the inductions necessary for ontology learning and evolution makes it such that statistical analysis ends up with largely arbitrary and unobjective results. Through the analysis of the results obtained in Section 4, the authors of this paper reached the same conclusion, and the possibilistic approach will be used to complement the statistical analysis. Being an unsupervised approach, the possibilistic approach is particularly suited for application to RDF streams and can accommodate for the incomplete nature of the data they provide. However, to the best of our knowledge, the possibilistic approach has only been applied for ontology validation, and its applicability to suggest new axioms from data analysis is either understudied or non-existent.

# 3 Property Axiom Scoring

## 3.1 Problem Definition

The application scenario of the TICO framework involves the existence of several streams of sensor data arriving in near real-time which are subject to transformations by data scientists, introducing new and unpredicted changes that are not in the original ontology. While the original application scenario was in the field of Predictive Maintenance, TICO is agnostic and can suggest changes to any ontology following changes in the individuals of the streams under analysis. TICO receives as input an ontology and a stream of individuals and outputs a set of evolutionary actions that can be executed over the original ontology to generate a new, evolved version of it. Consider, for example, a stream *S* of individuals describing familial relationships between people as seen in Figure 1.
<figure><img src="image_4.png" /><figcaption>**Figure 2.** Data structures concerning the property axiom testing process.</figcaption></figure>

The individuals are separated into three consequent groups, which correspond to different periods of analysis, at the end of which the ontology that represents these individuals may be updated. When analyzing the first group, one can conclude, for example, that the properties hasMother and hasFather should be functional, and the property hasBrother is both symmetrical and transitive. However, upon analyzing the second group of individuals, this assessment must be revised: The property hasBrother is still symmetrical, but no longer transitive, i.e., the brother of my brother is not necessarily my brother as well.

Now consider that, as society grows more open to different familial structures, individuals in the stream may now in fact have two legal parents of the same gender (represented by the third group in Figure 1); in order to accurately reflect reality, the application of the property *hasMother* must now accommodate for this necessity, and this change in how the property is used means its functionality is obsolete and should be reconsidered: The ontology should evolve to accommodate for the change in how the property is effectively used. This paper focuses on the changes introduced to TICO that go beyond the detection of new properties and into recognizing the property axioms that should be added with them. Unlike other logic-based approaches, TICO does so by analyzing positive and negative evidence regarding a set of known and predetermined rules – the property axioms – to ascertain if they should be added to the ontology. To better understand the property axiom testing process on which this paper focuses, consider Figure 2 and the descriptions that follow.

The analysis is triggered by the arrival of a new individual on the stream. The visualization provided by Figure 2 shows:

- The timeframe *t*, the period in which the stream *S* is analysed and from which to draw conclusions

- *Sw*, the sliding window upon which queries are executed

- The named individual *i*, which is composed by a set of RDF triples with the same Subject

- The use of *i* and its properties to instantiate query patterns for specific property axiom Ax(*P*)

- The query results being used to classify *i* as selective confirmation or counterexample for each Ax(*P*).

TICO receives as input a stream of individuals *S*, arriving continuously. Each named individual *i* in *S* is considered complete upon arrival (no new facts about *i* are analysed a posteriori, guaranteeing each individual is only analysed once). Because *S* is unbounded – with unknown start and ending points and potentially infinite – it is necessary, at some point, to assume sufficient analysis has been done and decisions that may result in ontology evolution can be made. With this in mind, the analysis the individuals delivered by *S* is executed during a particular period of time (or timeframe) *t*, at the
end of which an evaluation of the results is made to generate evolutionary actions. *t* represents a period in the stream that can be determined by the timestamps on the individuals themselves, a specific number of individuals or an actual time interval. From this setup, Definitions 1 and 2 can be elicited, such that:

**Definition 1.** (RDF Stream): An RDF stream is a (possibly infinite) sequence of triples <Subject, Predicate, Object>, in which: Subject is an IRI or blank node; Predicate is an IRI, and Object is a IRI, blank node or literal.

**Definition 2.** (Timeframe): An RDF stream can be divided into subsequences (*t*₁, *t*₂, ..., *t*ₙ,...) of triples called time-frames. Data about a particular individual *i* (i.e., triples for which *i* is the Subject) are bounded to a single timeframe, and arrive sequentially.

The consideration noted in Definition 2 stems from TICO's original application scenario – real-time sensor data – in which each individual encapsulates a sensor reading that took place at a particular moment, and no new data is added to previous readings. The remaining elements present in Figure 2 are described next. Each individual is only analysed once per timeframe, and no records about the individuals or the statistics they informed are maintained between timeframes.¹

**Sliding Window:** With the exception of Irreflexivity, all property axioms pertain to the relation between one individual and individuals other than itself. Since individuals arriving on a stream are otherwise lost after arrival, and keeping a permanent copy of each is not sustainable nor desirable, they are stored in a temporary structure upon which the analysis is executed – in a sliding window *Sw* of fixed length. In a first-in-first-out fashion, every time *S* delivers a new individual, it is added to *Sw*, until its maximum size is reached – i.e., the size of the sliding window is measured by the total number of individuals it can store, regardless of how many triples compose them. Afterwards, for each new arrival, the oldest individual is removed from *Sw* and forgotten.

**Property Constructor Axiom:** Property axioms Ax – or characteristics (Bühmann & Lehmann, 2012) – can be used to describe how the property must be employed. There are seven property axioms in OWL, namely: Functionality, Inverse Functionality, Transitivity, Reflexivity, Irreflexivity, Symmetry and Asymmetry. The definition of Reflexivity, however, is too strong to evaluate properly considering the constraints of this solution – requiring the analysis of all individuals at the Class level, which is not compatible with the property-oriented approach – and it is not in the scope of this paper. Focusing exclusively on the application of property axioms in Object Properties, Ax denotes one of the possible OWL property axioms among:

- FunctionalObjectProperty (*F*),

- InverseFunctionalObjectProperty (IF),

- TransitiveObjectProperty (*T*),

- SymmetricObjectProperty (*S*),

- AsymmetricObjectProperty (AS),

- IrreflexiveObjectProperty (IR)

and therefore:

$$Ax \in \{F, IF, T, IR, S, AS\}$$

All property axioms for a given property *P* can be expressed as first-order logic implications in the form:

$$Ax(P) : \forall i, \forall x_1, \dots, \forall x_n \\ B(P, i, x_1, \dots, x_n) \rightarrow H(P, i, x_1, \dots, x_n)$$

where *i* is an individual in the domain of a property *P*. For example, the functionality of *P* could be written as such an implication, in which the body *B* shows the simultaneous application of the same property more than once for the same individual – *B(P, i, x₁, x₂)* : *P(i, x₁)∧P(i, x₂)* – and the head *H* is the resulting implication that the ranges of said property must therefore be the same – *H(P, i, x₁, x₂)* : *x₁ = x₂*.

**Confirmation, Selective Confirmation and Counterexample:** In testing a property axiom hypothesis Ax(*P*), we assume the original ontology is consistent and does not neither entail Ax(*P*) nor its negation. Contrary to Tettamanzi's approach (Tettamanzi et al., 2017) substitutions occur not at the triple level, but at the individual level: A named individual *i* counts as either one confirmation or one counterexample, regardless of how many triples compose it.

For a named individual *a*₀ and a property *P*, substitutions *i*/*a*₀, *x*₁/*a*₁, ..., *x*ₙ/*a*ₙ for *B(P, i, x₁, ..., xₙ)* and *H(P, i, x₁, ..., xₙ)* may be found, such that:
- If there's at least one substitution for *B* and the negation of *H* under the Unique Name Assumption (UNA): The named individual *a*₀ is a **counterexample**;

- There is a substitution for both *B* and *H*, and no substitution for the negation of *H*: *a*₀ **selectively confirms** the hypothesis;

- There is a substitution for *B*, but no substitution for *H* nor for ∼*H*: *a*₀ does not contain enough information to selectively confirm nor deny the hypothesis, and therefore is a **weak counterexample**;

- If there is no substitution for *B*, *a*₀ is a confirmation, but not a selective one, and thus not considered and it is ignored.

This distinction between confirmation and selective confirmation follows from the selective confirmation principle (Scheffler & Goodman, 1972): A selective confirmation is a confirmation in which *i* is in the domain of *P*, i.e., selective confirmations are a subset of all possible confirmations.

Considering both UNA and the fact that only individuals in the domain of the property are counted, functionality, inverse functionality, irreflexivity and asymmetry cannot generate weak counterexamples – the imposed constraints guarantee that an individual is always classified as either a selective confirmation or counterexample – there are always substitutions for either *H* or ∼*H*. The unknown data brought upon by the constraints of stream and the sliding window does not affect the classification of individuals, as they are assumed to arrive complete – furthermore, counterexamples rely on more individuals being accessible in *Sw*, meaning there is no uncertainty regarding their correct classification.

On the other hand, the openness of the world and unknown facts about individuals in the range of the properties greatly affect the classification process for the transitivity and symmetry axioms – consider, for instance, that the individuals in the range of the properties may not be present in the sliding window when the query is executed, and therefore it is not possible to assess if that individual is also in the domain of the property (necessary for both axioms). Selective confirmations are indisputable, as their mere existence implies that the data was, in fact, complete. Counterexamples, on the other hand, imply proving a negative, making them particularly harder to identify under the constraints and assumptions of the methodology employed. As such, any substitutions that do not selectively confirm these axioms are treated as *weak counterexamples*.

**Constructor Query Pattern:** To classify *i* w.r.t. Ax(*P*), it is first necessary to ascertain if *P* is used by *i*. If that is the case, *i* is then used to instantiate a specific constructor query pattern – a SPARQL query which enforces the pattern associated with Ax(*P*). This query will enforce the substitutions mentioned previously, meaning reasoners are not employed. Code snippet 1 shows one possible query (for symmetry), in which *iURI* and *pURI* are known and correspond, respectively, to *i* and *P*. Each *i* is used to instantiate the query patterns only once, although it may influence the results of queries coming afterwards (as *Object*).

Code snippet 1. Query representing symmetry
{
  SELECT * WHERE {
  }
  SELECT (COUNT(?obj) as ?B) WHERE
  { <iURI> <pURI>> ?obj . }
}
{
  SELECT (COUNT(?obj) as ?H) WHERE
  { <iURI> <pURI>> ?obj.
    ?obj <URI> <iURI> . }
}
FILTER (?B = ?H) }

When the query is executed over *Sw*, any results it obtains classify *i* as a selective confirmation of Ax(*P*) (i.e., both the triple and its mirror are present). In this case, as it is symmetry, if the query does not obtain at least one result, then *i* is considered a *weak* counterexample of Ax(*P*).
<table><caption>**Table 1.** Example Individuals in a Sw and Respective Properties.</caption><thead><tr><th>ID</th><th>hasName</th><th>hasEvolutionGroup</th><th>hasType</th><th>hatches</th></tr></thead><tbody><tr><td>i<sub>1</sub></td><td>Onix</td><td>Onix &amp; Steelix Line</td><td>Rock, Ground</td><td>Mineral EG</td></tr><tr><td>i<sub>2</sub></td><td>Ditto</td><td></td><td>Normal</td><td>Ditto EG</td></tr><tr><td>i<sub>3</sub></td><td>Caterpie</td><td>Caterpie Line</td><td>Bug</td><td>Bug EG</td></tr><tr><td>i<sub>4</sub></td><td>Metapod</td><td>Caterpie Line</td><td>Bug</td><td>Bug EG</td></tr><tr><td>i<sub>5</sub></td><td>Tropius</td><td></td><td>Flying, Grass</td><td>Grass EG, Mineral EG</td></tr><tr><td>i<sub>6</sub></td><td>Bug EG</td><td></td><td></td><td>Caterpie, Metapod</td></tr><tr><td>i<sub>7</sub></td><td>Grass EG</td><td></td><td></td><td>Tropius, Maractus</td></tr><tr><td>i<sub>8</sub></td><td>Maractus</td><td></td><td>Grass</td><td>Grass EG</td></tr></tbody></table>

## 3.2 Property Constructor Analysis

The analysis of real-time data can unravel patterns regarding a property's usage that may hint at its formal definition. Each property axiom, per virtue of its definition, corresponds to a different pattern in the data (cf. Table 2). Consider the generic task computeAxiom, which aims to gather statistics about property axioms and is summarized in Algorithm 1.

$E^+$ and $E^-$ reflect, respectively, how many individuals are categorized as selective confirmations and strong counterexamples of $Ax(P)$ in $S$ during $t$. Support ($E^0$) is provided by the sum of $E^+$ and $E^-$ and reflects the total number of individuals in the domain of $P$.

Algorithm 1. computeAxiom

Input: $S$, $Sw$, $Ax$, $P$
Output: $E^+$, $E^-$
for each $i$ in $S$ do
    if $Sw$ is full then
        pop $Sw$
        push $i$ to $Sw$;
        $dP \leftarrow$ get distinct properties used by $i$;
    if $P$ in $dP$ then
        $QAx \leftarrow$ get query pattern for $Ax$;
        instantiate $QAx$ using $i$ and $P$;
        if $QAx$ has results in $Sw$ then
            increase $E^+$;
        else
            increase $E^-$;

Consider a $Sw$ with a size of 8, whose current individuals are described in Table 1. Individuals are complete but may contain references to individuals that are not in the $Sw$ at the time computeAxiom is executed.

Considering the potential Inverse Functionality of the hasEvolutionGroup property, i.e., IF(hasEvolutionGroup), the individuals are classified as follows:

- Samples: $i_1$, $i_3$ and $i_4$, as they use the property being analysed. $E^0 = 3$.

- Selective confirmations: $i_1$, as the domain of the property is only used once per value. $E^+ = 1$.

- Strong counterexamples: $i_3$ and $i_4$, as they have the same property value for different individuals. $E^- = 2$.

Individuals $i_2$, $i_5$, $i_6$, $i_7$ and $i_8$ do not use the property and are not used in the analysis of IF(hasEvolutionGroup). They may be used in the analysis of axioms for different properties.

Consider the potential symmetry of the hatches property. For $S$(hatches):

- Samples: $i_1$, $i_2$, $i_3$, $i_4$, $i_5$, $i_6$, $i_7$ and $i_8$. $E^0 = 8$.

- Selective confirmations: $i_7$, $i_8$, as both hatches($i_7$, $i_8$) and hatches($i_8$, $i_7$) are present. $E^+ = 2$.

- Weak counterexamples: $i_1$, $i_2$, $i_3$, $i_4$, $i_5$ and $i_6$ as they use the property in relation to other individuals, but whether those individuals also use it in relation to them is not made explicit in the data. $E^- = 6$.

The details of how each property axiom is defined and the formula to search for in the data are described in Table 2.
### 3.3 The Possibilistic Approach to Axiom Scoring

The work described in Tettamanzi et al. (2017) also introduces the concept of an ARI and its application to axiom scoring: Positive values suggesting acceptance, negative suggesting rejection and values close to zero indicating ignorance. For any $Ax(P)$, it is possible to calculate its necessity $N$: The degree to which the axiom is corroborated by the data while not being contradicted by it; and its possibility $\Pi$: The degree to which it is not contradicted by the data. ARI can thus be calculated using (Tettamanzi et al., 2014):

$$ \mathrm{ARI}(\mathrm{Ax}(P)) = N(\mathrm{Ax}(P)) + \Pi(\mathrm{Ax}(P)) - 1 $$

$N$ and $\Pi$ are calculated differently depending on the approach to the data being used. The original reasoning behind them can be found in Tettamanzi et al. (2014). For the purposes of this paper, the computation of $N$ and $\Pi$ depends on the relative weight given to selective confirmations and counterexamples.

If counterexamples are strong, a single counterexample is sufficient to quash $N$, regardless of how many selective confirmations are found. Ergo, counterexamples have more weight in the decision-making process than selective confirmations – when strong counterexamples are available, the strong form of $N$ and $\Pi$ is applied, as follows (Tettamanzi et al., 2014):

$$ N(\mathrm{Ax}(P)) = \begin{cases} 1, & \text{if } E_{\mathrm{Ax}(P)}^{-} = 0 \\ 0, & \text{if } E_{\mathrm{Ax}(P)}^{-} > 0 \end{cases} $$

and

$$ \Pi(\mathrm{Ax}(P)) = 1 - \sqrt{1 - \left(\frac{E_{\mathrm{Ax}(P)}^{+}}{E_{\mathrm{Ax}(P)}^{0}}\right)^{2}} $$

in which $E_{\mathrm{Ax}(P)}^{0}$ is the total number of individuals in the domain of $P$, corresponding to the sum of $E_{\mathrm{Ax}(P)}^{+}$ – the number of selective confirmations, – and $E_{\mathrm{Ax}(P)}^{-}$ – the number of counterexamples.

In the cases in which counterexamples are not sufficient to exclude a hypothesis – being weak counterexamples – the possibility of an axiom being true is directly related to the existence of selective confirmations. $N$ and $\Pi$ are calculated according to (Tettamanzi et al., 2014):

$$ N(\mathrm{Ax}(P)) = \sqrt{1 - \left(\frac{E_{\mathrm{Ax}(P)}^{-}}{E_{\mathrm{Ax}(P)}^{0}}\right)^{2}} $$

and

$$ \Pi(\mathrm{Ax}(P)) = \begin{cases} 0, & \text{if } E_{\mathrm{Ax}(P)}^{+} = 0 \\ 1, & \text{if } E_{\mathrm{Ax}(P)}^{+} > 0 \end{cases} $$

This second set of formulas ensures the decision is influenced more by the selective confirmations – with $\Pi$ being at its highest whenever selective confirmations are present. $N$ cannot be set to 0, as counterexamples are weak and the assumption that they are, in fact, actual counterexamples cannot be made under the Open World Assumption (OWA). For the sake of simplicity, the first set of formulas are considered the stronger forms and the second the weaker forms, after the strength of their respective counterexamples.

Table 3 summarizes which forms of $N$ and $\Pi$ (ARI form for brevity) to use per each property axiom. These reflect not only the assumptions made in Table 2, but also the constraints of the approach itself (under UNA, weak confirmations become strong for functionality and inverse functionality, for example).

Returning to the example in Table 1, not all individuals in the domain or range of the hatches property are equally available to be analysed – e.g., there is mention of an individual “Mineral EG” that is not among the individuals currently in Sw. One can see that this property relates individuals in a symmetrical fashion: $i_8$ relates to $i_7$ using this property and vice-versa. These would, indeed, count as selective confirmations of $S$(hatches) (and for $T$(hatches)). However, all individuals in the sliding window use the property, which results in the data in Table 4.

By not having information about individuals such as Mineral EG and Ditto EG, it is not possible to have all the data needed for the remaining individuals to count as selective confirmations. In the cases of $T$ and $S$, incomplete data may
<table><caption>**Table 2.** Definitions, Formula, Selective Confirmation and Counterexamples for Each Property Axiom.</caption><thead><tr><th>Aw</th><th>Definition</th><th>Formula</th><th>Selective Confirmation (E<sup>+</sup>)</th><th>Counterexamples (E<sup>-</sup>)</th><th>Assumptions</th></tr></thead><tbody><tr><td>F</td><td>P can have one and only one value per i. P may be employed more than once by i provided the ranges are the same individual.</td><td>∀i, y<sub>1</sub>, y<sub>2</sub> : P(i, y<sub>1</sub>) ∧ P(i, y<sub>2</sub>)<br>→ y<sub>1</sub> = y<sub>2</sub></td><td>i for which P has either only one value.</td><td>i for which P is used more than once and the ranges of P are distinct. Counterexamples of F(P) are strong counterexamples.</td><td>Partial closure of the world through Unique Name Assumption (UNA) allows for the existence of strong counterexamples.</td></tr><tr><td>IF</td><td>P cannot have the same entity on its range for more than one i (e.g., a unique ID cannot be shared).</td><td>∀i<sub>1</sub>, i<sub>2</sub>, y : P(i<sub>1</sub>, y) ∧ P(i<sub>2</sub>, y)<br>→ i<sub>1</sub> = i<sub>2</sub></td><td>i for which the value of y has not been the range of P in any previously analyzed individual in Sw.</td><td>i for which the value of y was found in the range of P of other previously seen individuals. Counterexamples of IF(P) are strong counterexamples.</td><td>Closed World Assumption allows for the elevation of non-confirmations to weak counterexamples.</td></tr><tr><td>T</td><td>If P holds between any two individuals in a sequence, it holds between all individuals of that sequence.</td><td>∀i, y, z : P(i, y) ∧ P(y, z) → P(i, z)</td><td>i for which P has propagated between any set of three individuals i, y, z, for which a P(i, z) exists.</td><td>i for which P does not propagate between any set of three individuals i, y, z. Non-confirmations of T(P) are considered weak counterexamples, as a strong counterexample would have to prove ¬P(i, z).</td><td>Partial closure of the world through UNA allows for the existence of strong counterexamples.</td></tr><tr><td>IR</td><td>Characteristic of a property that cannot relate i with itself. The domain and range of the P must not be the same individual.</td><td>∀i : ¬P(i, i) or ∀i<sub>1</sub>, i<sub>2</sub> : P(i<sub>1</sub>, i<sub>2</sub>)<br>→ i<sub>1</sub> ≠ i<sub>2</sub></td><td>i in which P is used to relate with individuals other than itself.</td><td>i in which P is used to relate with itself. Counterexamples of IR(P) are strong counterexamples.</td><td>Closed World Assumption allows for the elevation of non-confirmations to weak counterexamples.</td></tr><tr><td>S</td><td>Property which is its own inverse: meaning that if P(a, b), then P(b, a) must also be true.</td><td>∀i, y : P(i, y) → P(y, i)</td><td>i in which the range of P relates back to the individual through P.</td><td>i for which the range of P does not relate back to the individual through P. Non-confirmations of T(P) are considered weak counterexamples, as a strong counterexample would have to prove ¬P(y, i).</td><td>Partial closure of the world through UNA allows for the existence of strong counterexamples.</td></tr><tr><td>AS</td><td>Property which cannot be its own inverse: if P(a, b), then P(b, a) cannot hold true.</td><td>∀i, y : P(i, y) → ¬P(y, i)</td><td>i for which the range of P does not relate back to the individual through P.</td><td>i for which the range of P relates back to the individual through P. Counterexamples of AS(P) are strong counterexamples.</td><td>Partial closure of the world through UNA allows for the existence of strong counterexamples.</td></tr></tbody></table>
<table><caption>**Table 3.** Acceptance/Rejection Index (ARI) Form to Apply for Each Property Axiom.</caption><thead><tr><th>Property Axiom</th><th>ARI Form</th></tr></thead><tbody><tr><td>Functionality</td><td>Strong</td></tr><tr><td>Inverse functionality</td><td>Strong</td></tr><tr><td>Transitivity</td><td>Weak</td></tr><tr><td>Symmetry</td><td>Weak</td></tr><tr><td>Asymmetry</td><td>Strong</td></tr><tr><td>Irreflexivity</td><td>Strong</td></tr></tbody></table>

<table><caption>**Table 4.** Axiom Scoring for the hatches Property Using the Strong Forms of N and Π.</caption><thead><tr><th>Ax(hatches)</th><th>E<sup>+</sup></th><th>E<sup>-</sup></th><th>N</th><th>Π</th><th>Acceptance/Rejection Index (ARI)</th></tr></thead><tbody><tr><td>F</td><td>6</td><td>2</td><td>0</td><td>0.34</td><td>-0.66</td></tr><tr><td>IF</td><td>5</td><td>3</td><td>0</td><td>0.22</td><td>-0.78</td></tr><tr><td>AS</td><td>3</td><td>5</td><td>0</td><td>0.07</td><td>-0.93</td></tr><tr><td>IR</td><td>8</td><td>0</td><td>1</td><td>1</td><td>1</td></tr></tbody></table>

<table><caption>**Table 5.** Axiom Scoring for the hatches Property Using the Weak Forms of N and Π.</caption><thead><tr><th>Ax(hatches)</th><th>E<sup>+</sup></th><th>E<sup>-</sup></th><th>N</th><th>Π</th><th>Acceptance/Rejection Index (ARI)</th></tr></thead><tbody><tr><td>T</td><td>5</td><td>3</td><td>0,93</td><td>1</td><td>0.93</td></tr><tr><td>S</td><td>2</td><td>6</td><td>0,48</td><td>1</td><td>0.66</td></tr></tbody></table>

lead to false negatives, and selective confirmations are much harder to find as they require more individuals to be on the same sliding window. When using the weaker forms to compute ARI for T and S, the results reflect the potential for those axioms to be present (see Table 5).

While ARI can be updated with each new individual on the stream, it only influences the ontology evolution process at the end of *t*. Algorithm 2 shows how the decision to include or exclude a property axiom from an ontology can be made, considering the existence of different thresholds for the weak and strong forms of ARI (wf<sub>t</sub> and sf<sub>t</sub>, respectively).

**Algorithm 2.** ARI Decision and evolving ontology

Algorithm is executed at the end of *t*.
Input: ontology, set of <Ax(P), E<sup>+</sup>, E<sup>-</sup>>, sf<sub>t</sub>, wf<sub>t</sub>
Output: ontology (evolved)
for each Ax(P) do
    if Ax one of { T, S } then
        compute N(Ax(P)) using weak form;
        compute Π(Ax(P)) using weak form;
        t ← wf<sub>t</sub>;
    else
        compute N(Ax(P)) using strong form;
        compute Π(Ax(P)) using strong form;
        t ← sf<sub>t</sub>;
        ARI(Ax(P)) ← N(Ax(P)) + Π(Ax(P)) - 1;
    if ARI(Ax(P)) >= t then
        add Ax(P) to ontology;
    else
        remove Ax(P) from ontology;

## 3.4 Applying ARI to Evolving Ontologies

Considering the ontology evolution application scenario, in which a potential new version (or at least for some of its axioms) is created at the end of each *t*, any conclusion reached over the application of the queries must be made in the context of available knowledge – i.e., previous known versions of the axioms in the ontology. As such, we use a weighted
average between the newly calculated ARI and the previous state of the axiom, which will be referred to as Evolving ARI (ARI<sub>e</sub>).

$$ \text{ARI}_e(\text{Ax}(P)) = \begin{cases} \text{ARI}(\text{Ax}(P)), & t = 0 \\ d(\text{Ax}(P))_{t-1} * w_p + \text{ARI}(\text{Ax}(P)) * (1 - w_p), & t \ge 1 \end{cases} $$

in which:

(1) ARI calculated during *t*;

(2) $d(\text{Ax}(P))_{t-1}$ is one of [0,1], depending on whether the axiom in question was (or was not) missing from the previous known version of the ontology;

(3) $w_p$ is the relative weight of previous knowledge [0-1]. Lower weights should allow for more versatility in the evolutionary process, favouring new axioms, while higher ones value a more conservative approach.

A $w_p$ of 0 would imply that previous knowledge does not affect the current decision-making, but a $w_p$ of 1 would equally mean that no change to the axioms in the ontology would ever be possible, regardless of how much evidence for it is found.

Both ARI and ARI<sub>e</sub> are on the [-1,1] interval, with positive results suggesting acceptance of the axiom and the inverse for negative results. When it comes to decision-making, however, this may not be sufficient (maybe not all positive results are strong enough to force asserting an axiom, and it may be interesting to allow for errors in the data to exist). For strong-form using properties, the decision *d* to accept or reject an axiom is informed by:

$$ d(\text{Ax}(P)) = \begin{cases} \text{accept}, & E_{\text{Ax}(P)}^{+} > \text{cf}_t \\ \text{accept}, & E_{\text{Ax}(P)}^{+} \le \text{cf}_t \text{ and ARI} > \text{sf}_t \\ \text{reject}, & \text{otherwise} \end{cases} $$

in which $E_{\text{Ax}(P)}^{+}$ is the percentage of selective confirmations (w.r.t. the Support), $\text{cf}_t$ the minimum percentage of selective confirmations required, and $\text{sf}_t$ is the threshold to be applied for the strong form of ARI computation. For weak-form using properties (i.e., *T* and *S*) the decision is informed exclusively by ARI and a minimum threshold $w_f$. Since the existence of counterexamples does not have the same weight as for the strong form, the reasoning to include the percentage of support in the decision-making process does not apply. As such, axiom acceptance is informed by:

$$ d(\text{Ax}(P)) = \begin{cases} \text{accept}, & \text{ARI} > w_f \\ \text{reject}, & \text{otherwise} \end{cases} $$

As *F* and *T* axioms are incompatible, and *F* is computed on stricter terms (using the stronger-form), *F* is considered to take precedence over *T* in case of both being flagged for inclusion. As such, whenever the algorithm concludes that the same property could be both *F* and *T*, it will only classify it as *F*. Similarly, an axiom cannot be simultaneously *T*, *S* and *IR*. In these cases, precedence is given to *T* and *S*.

The following sections will illustrate the application the possibilistic approach to axiom scoring in two different scenarios:

- **Experiment I:** In this experiment, ARI will be compared to traditional information retrieval metrics to ascertain its applicability to axiom scoring in an RDF stream scenario. For this purpose, existing ontologies for which the property axioms are known will be used. This experiment will also assess the effects of different sliding window sizes in the proper classification of individuals as selective confirmations or counterexamples.

- **Experiment II:** The experiment will be conducted using an ontology automatically generated from publicly available datasets to establish the applicability of the solution in an ontology evolution/learning scenario. Information regarding property axioms is not known *a priori*, and the suitability of the proposed axioms will be analysed by employing a reasoner and verifying if inconsistencies are introduced. This experiment will show the suitability of the solution in cases in which the data is both potentially incomplete and with errors. Furthermore, the individuals analysed will be segregated into different, pre-established timeframes, allowing for the application of Evolving ARI between them.
<table><caption>**Table 6.** Properties and Their Respective Axioms by Class.</caption><thead><tr><th></th><th>Class</th><th>Property</th><th>Ax</th></tr></thead><tbody><tr><td rowspan="10">CMT (Ontology Alignment Evaluation Initiative 2023, 2023)</td><td rowspan="4">Paper (4301 individuals)</td><td>hasAuthor</td><td>F</td></tr><tr><td>hasDecision</td><td>F</td></tr><tr><td>readByMetaReviewer</td><td>F</td></tr><tr><td>rejectedBy</td><td>F</td></tr><tr><td rowspan="6">Person (4255 individuals)</td><td>addedBy</td><td>F</td></tr><tr><td>addProgram CommitteeMember</td><td>IF</td></tr><tr><td>assignedByReviewer</td><td>F</td></tr><tr><td>assignExternalReviewer</td><td>IF</td></tr><tr><td>rejectPaper</td><td>IF</td></tr><tr><td>writePaper</td><td>IF</td></tr><tr><td>writeReview</td><td>IF</td></tr><tr><td rowspan="2">WINE (How does it work? (OWL Example Wine Agent), 2003)</td><td>Region (36 individuals)</td><td>adjacentRegion</td><td>S</td></tr><tr><td>Region (as range)</td><td>locatedIn</td><td>T</td></tr><tr><td>Plant (Bruskiewich et al., 2002; Oregon State University, 2014)</td><td>Thing (82 individuals)</td><td>part_of/has_part</td><td>T</td></tr></tbody></table>

## 4 Experiment I: Effects of Sliding Window Size and Suitability of ARI for Axiom Scoring

To establish the strength of the devised solution, experiments were first conducted against a dataset in which some property constructors were previously known. With the ontologies known and the datasets curated, there is no expectation of counterexamples to be found for the assertions present in the ontology.

For a better understanding and discussion of the results, they have been separated according to the following questions:

(1) What should be the size of the sliding window in order to effectively learn property axioms and how does the number of samples (i.e., individuals that use the property under scrutiny) in the stream influence it? Furthermore, the following subquestion will be investigated:

(a) How does ARI compare with precision, recall and f-measure? How does it compare to the application of selective axiom confirmations exclusively?

(2) How well do the weak forms of N and Π compensate for the difficulty in identifying selective confirmations under OWA?

The experiments described below were carried out using the set of ontologies in Table 6, which were selected for their property axioms and population sizes.

Each populated ontology was split into samples of 3000 random individuals, which are then sequentially subjected to the analysing queries. To simulate an RDF stream scenario, the individuals of the ontology are queued (and queued when necessary) into the sliding window one at a time. Queries are executed over the sliding window, guaranteeing they never have access to all existing individuals in the ontology simultaneously and therefore operate under the original restrictions of the TICO solution. Because it is not possible for one property to employ all property axioms simultaneously, we can restrict the analysis to only individuals that make use of the properties shown in Table 6 for performance reasons – and individuals that work as confirmations for those properties can work as counterexamples for others.

For the purposes of this work, a modified version of the queries described in Bühmann & Lehmann (2012) are used as confirmation queries (CQs), in which the differences account for the streaming nature of the use case and the granularity of the search (here at the individual level and not at the triple level). Two different types of queries can be used for each property axiom: The CQs, which ascertain if a property axiom could be present, and the negation queries (NQ), which ascertain the opposite. Each class of query (CQ or NQ) provides its own confusion matrix, and the meaning of their solutions varies depending on whether the true class corresponds to the existence or non-existence of the axiom. This reasoning is illustrated in Table 7.

The CQ looks for positive cases of functionality: If a given individual is compatible with the axiom, either by having only one use of the property or the object being a duplicate. An individual that can be selected with this query is a selective confirmation of the functionality axiom. If the property is indeed functional – i.e., if the true class in Table 7 is AXIOM – then this result is a true positive. On the other hand, if the axiom was present but the query does not yield any results, it is a false negative. If the true class is $\neg$AXIOM – i.e., it is known that the functionality axiom is not present – and the
<table><caption>**Table 7.** Confusion Matrix and Respective Query Results.</caption><thead><tr><th rowspan="2">Applied query</th><th rowspan="2"></th><th rowspan="2">Has Solution?</th><th colspan="2">Target</th></tr><tr><th>AXIOM</th><th>¬AXIOM</th></tr></thead><tbody><tr><td rowspan="4"></td><td rowspan="2">CQ</td><td>Yes</td><td>True Positive (TP)</td><td>False Positive (FP)</td></tr><tr><td>No</td><td>False Negative (FN)</td><td>True Negative (TN)</td></tr><tr><td rowspan="2">NQ</td><td>Yes</td><td>False Positive (FP)</td><td>True Positive (TP)</td></tr><tr><td>No</td><td>True Negative (TN)</td><td>False Negative (FN)</td></tr></tbody></table>

CQ: confirmation query; NQ: negation query.

query returns a result, it is a false positive. Following the same reasoning, an empty set here is the correct result for the individual and therefore a true negative.

The NQ, on the other hand, looks for explicit counterexamples: For an individual to result in a non-empty set when queried, it must definitively have at least two uses of the property, and their objects be distinct. If the true class in Table 7 is AXIOM and the NQ returns a non-empty set of solutions, it must forcefully be a false positive – there should not have been any solutions for the query, as functionality is present. If it returns an empty set, it is a correct identification and a true negative. In the same vein, the true class is ¬AXIOM – and the query has results, it is doing so correctly and, therefore, a true positive. If it returns none – claiming the axiom is present when it should not be – it accounts for a false negative.

Code snippet 2 and Code snippet 3 show one possible difference between confirmation and NQs, using the functionality axiom as an example. Each query is executed for each individual being tested and generates a result set with a size of either 0 or 1 query solutions.

Code snippet 2. Confirmation Query for Functionality
SELECT ?o1 WHERE
{
  <iURI> <pURI> ?o1.
  FILTER NOT EXISTS
    { <iURI> <pURI> ?o2.
      FILTER (?o1 = ?o2)
    }
}

Code snippet 3. Negation Query for Functionality
SELECT ?o1 WHERE}
{
  <iURI> <pURI> ?o1. <iURI> <pURI> ?o2.
  FILTER (?o1 != ?o2)
}

Traditional information retrieval statistics, e.g., precision, recall and f-measure are computed as follows:

$$ 
\begin{align*}
\text{precision} &= \frac{\text{TP}}{\text{TP} + \text{FN}} & \text{recall} &= \frac{\text{TP}}{\text{TP} + \text{FP}} \\
\text{f-measure} &= \frac{2 * \text{precision} * \text{recall}}{\text{precision} + \text{recall}}
\end{align*}
$$

To ascertain how 'quickly' and effectively the queries can identify the possibility/probability of an axiom being present for a given individual, three different sliding windows sizes are analysed: 10, 50 and 100 individuals (corresponding,
<table><caption>**Table 8.** Number of Individuals on the Stream to Analyse, Window Sizes, and Percentage of Support in Each Sample for Each of the Studied Properties.</caption><thead><tr><th>Variable</th><th>Value</th></tr></thead><tbody><tr><td>individuals analysed</td><td>2150</td></tr><tr><td>w<sub>s</sub> (size of Sw)</td><td>10/50/100</td></tr><tr><td>Property</td><td>% of Support</td></tr><tr><td>readByMetaReviewer</td><td>100%</td></tr><tr><td>rejectedBy</td><td>≈50%</td></tr><tr><td>addedBy</td><td>≈25%</td></tr></tbody></table>

roughly, to 0.2%, 2.3% and 4.7% of all individuals in the sample, respectively). Theoretically, the more individuals in the sliding window that can be used to answer a query, the more precise the classification of each individual as a selective confirmation or counterexample should be. However, for performance reasons, it is interesting to see if reliable conclusions can be made from as little data as possible.

While all properties used by each class were studied, for the sake of brevity, we will focus on presenting the results for a property from a class with a high variance in the usage of its properties – i.e., not all individuals of the same type will use the same properties – and those for a class with lower variance, in which all individuals always use the same properties.

## 4.1 Section I – Effects of Window Size and Relevance of Individuals in Axiom Support

This section analyses the effects of both the window size and the influence of support in the sample. As previously explained, for an individual to constitute support, it must be the domain of the property being investigated. However, even if a stream is comprised only by individuals of the same type, there is no guarantee that all of them will employ the same properties. For the following experiments, two classes from the CMT ontology were selected – Paper and Person – as the first always employs all its properties and the second has more variety to it, with some properties being used in a very low fraction of its individuals. To ascertain the influence of the window size ($w_s$) of the Sw, three different values (of 10, 50 and 100 individuals) are applied to each of the classes.

Considering that the dataset is clean, and no counterexamples can be found, the results in favour of an axiom are fairly evident regardless of the window size applied (perfect precision combined with necessity, possibility, and ARI of 1). However, it is important to investigate the more interesting cases in which counterexamples are indeed possible, by analysing the evolution of the metrics in the cases where an axiom is known not to be present.

The values presented in Table 8 are applied on the following experiments, assessing how the relevance of each individual for support affects the evolution of the metrics and the potential results. The following graphs show the evolution of specific measure in function of the number of individuals analysed.

The readByMetaReviewer property does not feature any of the explored property axioms, but it is present in all individuals of the Paper class. The study of the evidence for and against the IF axiom for this property shows how the proper identification of each individual as a selective confirmation or a counterexample is affected by the changes in window size – that as more individuals are analysed, it becomes apparent that the same property is used by more individuals with the same value – and the number of counterexamples starts to increase. Most of the changes occur when analysing the first 60 individuals out of the sample, as shown in Figure 3, with most individuals (around 94%) being incorrectly categorized as selective confirmations due to lack of information to the contrary. However, by increasing $w_s$ to 50, the categorization improves significantly: The number of selective confirmations lowers to around 96%, and to 90% when $w_s$ is increased to 100. With recall being 100% regardless of the window size chosen, Figure 4 compares the evolution of precision, showing similar improvements although with a high propensity for misclassification (stabilizing around 10%).

ARI is always steadily negative, with the improvements in the classification of individuals changing how negative it skews, improving from -0.36 to -0.71, as shown in Figure 5 (featuring only the first 60 individuals, after which the metrics stabilize). The metrics evolve differently when not all individuals in the stream are considered support.

Consider the results in Figure 6, which were obtained when searching for inverse functionality of the rejectedBy property, which is used by 1279 individuals of the Paper class (≈50%).

Selective confirmations for the IF axiom start high (at 100%), as a single individual with a single use of the property cannot be flagged as a counterexample. After analysing 60 individuals, counterexamples account for more than 70% of the samples; 80% after 120, and their number finally stabilizes around 85% as the analysis progresses. With a sliding window of 50, there should be more available evidence for each query to ascertain if the axiom is present. This seems indeed to be the case: While the 70% threshold obtained with a $w_s$ of 10 is equally obtained after analysing around 60 individuals,
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Evolution of the selective confirmations of IF for the readByMetaReviewer property with $w_s$ of 10, 50 and 100.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Figure 4.** Evolution of the Precision of IF for the readByMetaReviewer Property with $w_s$ of 10, 50 and 100.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Figure 5.** Evolution of N, Π and acceptance/rejection index (ARI) of IF for the readByMetaReviewer property with $w_s$ of 10 (first graph), 50 (second graph) and 100 (third graph).</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 6.** Evolution of selective confirmations of IF for the rejectedBy property with $w_s$ of 10, 50 and 100.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Figure 7.** Evolution of precision, recall and *f*-measure of IF for the rejectedBy property with *w*<sub>s</sub> of 10 (first graph) and 50 (second graph).</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**Figure 8.** Evolution of *N*, *II* and acceptance/rejection index (ARI) of IF for the rejectedBy Property with *w*<sub>s</sub> of 10.</figcaption></figure>

with a *w*<sub>s</sub> of 50 the 70% threshold is reached after analysing only 16 samples. The percentage of counterexamples also stabilizes at a higher point (at around 94%, after circa 75 individuals). Increasing *w*<sub>s</sub> to 100 shows very minor increases in both speed and accuracy. The number of counterexamples on the sample reaches the 70% threshold earlier than with *w*<sub>s</sub> of 50 – after 12 individuals – and stabilizing at a negligibly slightly higher point – at 97%.

Precision, recall and *f*-measure show similar evolution patterns as those of support, as seen in the first graph of Figure 7. All metrics start low, with a quick but unsteady increase until around 50 individuals have been analysed, and equally stabilizing as the analysis progresses.

Precision peaks at around 85%, as expected from the results presented in Figure 6. Recall stabilizes around 45% and *f*-measure at 60%. With a similar evolution to that of the support for the same window size, as illustrated in the second graph of Figure 7, similar levels of precision, recall and *f*-measure are obtained but with their values stabilizing higher and later (*f*-measure reaching 60% after circa 40 samples and 65% after 100). This effectively shows that by having a bigger sliding window – i.e., by having more individuals available for each query to search in – it is possible to track more nuances in the data and better classify each individual as a selective confirmation or counterexample. In further increasing *w*<sub>s</sub> to 100, no significant improvements are made. Precision, recall and *f*-measure stabilize faster, but improvements account for little more than 1%.

If one were to trust these metrics by themselves, it could be concluded that the approach can identify if the property is not IF with a certainty of 65%. Here, necessity, possibility and ARI can provide a faster and more complete idea of the classification that must be done when giving the proper weight to the counterexamples. The following Figures illustrate the evolution of ARI using the same three window sizes as before.

Figure 8 shows necessity starting at 1, as only one individual has been analysed, and it is not a counterexample. However, as the second individual provides a counterexample, it immediately drops and stays at 0. Possibility, as determined by the number of selective confirmations, suffers a gradual loss as less and less selective confirmations are found in the data, and stabilizes very close to 0 (but with a positive value, as individuals that do not explicitly contradict the axiom can still be found) after around 50 individuals.

ARI, as a function of the other two metrics, shows a similar evolution from 50 individuals onwards, stabilizing very close to -1, which strongly advocates for axiom rejection, as seen in Figure 9.
<figure><img src="image_11.png" /><figcaption>**Figure 9.** Evolution of *N*, Π and acceptance/rejection index (ARI) of IF for the rejectedBy Property with *w*<sub>s</sub> of 50.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>**Figure 10.** Evolution of *N*, Π and acceptance/rejection index (ARI) of IF for the rejectedBy Property with *w*<sub>s</sub> of 100.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Figure 11.** Evolution of confirmations of IF for the property addedBy with *w*<sub>s</sub> of 10, 50 and 100.</figcaption></figure>

There are no discernible changes in the evolution of necessity, possibility and ARI when the window size is increased, as the number of selective confirmations and counterexamples is not as relevant for these metrics as the mere presence of counterexamples is. The metrics support the reasoning that a bigger window provides a better classification of each sample as either a selective confirmation or a counterexample, but only to a certain extent.

Similar to support and information retrieval metrics, Figure 10 displays how ARI evolves slightly faster but is not significantly improved. Unlike the Paper class previously explored, Person has even more variance in the application of its properties – e.g., the property addedBy is used by 734 of the 3000 individuals studied (≈25%), while rejectPaper is used only by 4 of them (≈0.1%) – although it is relevant to note that rejectPaper is the inverse property of rejectedBy, which is more widely used. While this change in frequency affects the evolution of the metrics being studied, the results follow the same trends as before: By allowing the queries to access more individuals, metrics are improved, but only until a certain point. The speed at which they improve, however, is indeed affected by the variety in the data, as shown in Figure 11.

The results show that if potential variations in use of the properties can be expected, increasing *w*<sub>s</sub> allows for better classification as selective confirmation or counterexample. The same conclusion can be drawn from the analysis of the differences in precision, recall and *f*-measure for the three values of *w*<sub>s</sub>, shown in Figure 12.

Furthermore, it is important to note that for the smaller values of *w*<sub>s</sub>, the evolution of the metrics is not monotonic (see the first graph of Figure 12). This suggests a lower *w*<sub>s</sub> can be detrimental when parsing a large number of individuals.

Finally, the changes in the evolution of necessity, possibility and ARI follow the information retrieval ones, by showing how a small window size for a type with high property variety takes longer to stabilize, as seen in Figure 13.

However, while there are significant improvements when *w*<sub>s</sub> is increased from 10 to 50, the same cannot be said from increasing it from 50 to 100 (much like in the studies for the Paper class), as seen in Figure 14.

In conclusion, in the cases where there is a very limited number of counterexamples, ARI may never reach its lower threshold; but remains reliably negative, even when *f*-measure is at its lowest – showing that the possibilistic approach is
<figure><img src="image_14.png" /><figcaption>**Figure 12.** Evolution of precision, recall and f-measure of IF for the property addedBy with w_s of 10 (first graph), 50 (second graph), and 100 (third graph).</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>**Figure 13.** Evolution of N, Π and acceptance/rejection index (ARI) of IF for the property addedBy with w_s of 10.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>**Figure 14.** Evolution of N, Π and acceptance/rejection index (ARI) of IF for the Property addedBy with w_s of 50 (first graph) and 100 (second graph).</figcaption></figure>
<figure><img src="image_17.png" /><figcaption>**Figure 15.** Evolution of *N*, Π and acceptance/rejection index (ARI) of IF for hasAuthor with *w*<sub>s</sub> of 10 (first graph) and 50 (second graph).</figcaption></figure>

indeed robust and applicable when scoring axioms in streams and with limited data, and more so than a strictly probability-based one. It nonetheless benefits from increased number of counterexamples, implying that a *w*<sub>s</sub> of 50 may provide sufficiently reliable results while also accounting for the negative effects of variations in property use.

Depending on the completion of the data, the incompleteness of the ontology, or simply because no such cases have
ever been documented – the same sample can easily selectively confirm multiple property axioms. Consider, for example,
that falsifying both *F* and IF axioms rely on the existence of more than one sample, or at least that the one sample uses the
same property more than once. This is an unreasonable expectation to have about the data, and any decision to include the
axioms needs to consider the fact that absence of evidence is not evidence of absence.

Since a bigger window size allows for better categorization of samples, it is possible to see the effect in evolution of
IF's ARI for the functional property *hasAuthor*. Consider the difference between the graphs in Figure 15.

Not only is the number of counterexamples very low, but they also take some time to occur in the stream. This means that while there are counterexamples, since there are so many sequential selective confirmations – and they continue even after a few counterexamples are encountered – *N* may drop to 0, and Π remains high to accommodate for potential errors in data. Therefore, while ARI skews negative, it remains relatively close to 0 (full uncertainty). However, it is once again interesting to note the benefits of the bigger window in the correct categorization of samples, as it allows for the negative ARI to be reached sooner (and more negative) as the window size increases.

Again, it is important to reiterate that this is a fortunate case: While the counterexamples may have taken longer to arrive and be few, they were still identifiable. When such is not the case, it has to be considered that just because a property is only used once per sample, it does not necessarily mean that it must certainly be *both* functional and inverse functional (although it is possible to be both at once) – even if their Π, *N* and ARI are always at 1.

## 4.2 Section II – Using the Weak Forms of Possibility and Necessity

Transitivity was studied using the Wine and Plant ontologies, with similar results. Since the Wine ontology had more available individuals with transitive properties (a total of 82), the following results reflect those exclusively.

Figure 16 shows how the lack of explicit confirmations affects the support for an axiom. It was necessary to analyse at least 30 individuals until a confirmation could be found, with the number increasing steadily until it reaches around 15%. Information retrieval metrics, as seen in Figure 17, also illustrate the clear lack of selective confirmations, which informed the decision to apply the weak forms of Π and *N*.

By giving more weight to the very hard to find selective confirmations than to the very easily incorrectly identified counterexamples, Figure 18 shows it is possible to obtain a positive, albeit conservative, ARI.

Should the strong form of possibility and necessity been applied instead, ARI would tend towards extremely negative (circa -1), even if the possibility was seen increasing (very) slowly.
<figure><img src="image_18.png" /><figcaption>**Figure 16.** Evolution of confirmations and counterexamples of *T* for the property located in *w*<sub>s</sub> of 50.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>**Figure 17.** Evolution of information retrieval metrics of *T* for the property located in *w*<sub>s</sub> of 50.</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>**Figure 18.** Evolution of the weak forms of Π, *N* and acceptance/rejection index (ARI) of *T* for the property located in *w*<sub>s</sub> of 50.</figcaption></figure>

## 4.3 Discussion

Of the three options in window size studied – of 10, 50 and 100 individuals – the results show that there is a significant improvement between going from 10 to 50, but hardly any from 50 to 100. We find 50 individuals to be a good middle ground for these datasets; while the results can be improved by employing bigger sliding windows, they also validate our assumption that it is possible to efficiently learn property axioms from a relatively small number of samples.

The observation of the previous results show that the computation of ARI contains more information than that just the percentage of selective confirmations and could potentially be used in its place. Additionally, ARI shows better performance than traditional information retrieval metrics, achieving sharper results and considerably earlier. However, since the existence of a single counterexample immediately skews ARI towards negative, $E_{\lambda x(P)}^{+}$ cannot be altogether excluded, especially considering that potential errors in data could be classified as counterexamples. Furthermore, the results suggest that some axioms benefit from the application of a higher threshold for ARI than others.

For axioms that are relatively easy to falsify, but not so easy to prove, like *F* and *IF*, a higher threshold for ARI should allow for some errors in the data while still strongly advocating for the inclusion of said axioms. On a window size of 50 individuals, we argue that ARI should be of 1 for an axiom to be considered for inclusion, but have a proportion of $E_{\lambda x(P)}^{+}$ of at least 95%. Alternatively, for axioms for which there may be an overabundance of negatively identified counterexamples and therefore make use of the weak forms of *N* and *Π*, a lower threshold for ARI should be considered: Allowing an axiom to be considered for inclusion even though there is relatively little evidence for it. For these, we propose a threshold for ARI of around 0.5.
<table><caption>**Table 9.** Values and Thresholds Employed in the Experiments.</caption><thead><tr><td>Variable</td><td>Description</td><td>Value</td></tr></thead><tbody><tr><td>cf<sub>t</sub></td><td>Percentage of selective confirmations w.r.t. support</td><td>95%</td></tr><tr><td>sf<sub>t</sub></td><td>Threshold for the Strong form of ARI</td><td>0.5</td></tr><tr><td>wf<sub>t</sub></td><td>Threshold for the Weak form of ARI</td><td>0.5</td></tr><tr><td>w<sub>s</sub></td><td>Sliding window size</td><td>50</td></tr></tbody></table>

There is a second part to the discussion of the existence of counterexamples: Axioms that are the opposite of one another. While a property can be both functional and inverse functional at the same time, it cannot be both symmetrical and asymmetrical. Furthermore, while a selective confirmation of S is a counterexample of AS, a selective confirmation of either does not necessarily mean that the property axiom must be present; any decision-making processes need to take this in consideration. This is especially evident in the case of property hasDecision of the CMT ontology, which is functional, but shows N, Π and ARI of 1 for F, IF, AS and IR.

# 5 Experiment II: Accommodating for Errors in Data and the Effects of Previous Knowledge in Axiom-Inclusion Decisions

The following experiment pertains to shows the applicability of the solution for the purpose of ontology enrichment/evolution. This is done by analysing a stream of individuals in which not only new properties are added over time, but the known properties are also applied in different ways to describe the data. As such, a suitable dataset needs to support the following: (1) Have several properties that connect the individuals in diverse ways, such that all different property axioms being studied could be tentatively discovered, (2) the application of said properties should change over time, as the domain naturally evolves, (3) the property axioms are not previously known.

The goal of this experiment is to answer the following questions:

1. When combining ARI and Support, are any inconsistencies introduced? To which extent? Are the inconsistencies the result of incorrect data or valid, but marginal uses of the properties?

2. If previous knowledge is considered in the decision to include/exclude a property axiom, does it prevent or increase the amount of inconsistencies introduced?

The ontologies in use pertain to the Pokémon domain as it is described in Wikidata (Wikidata, n.d). Pokémon is a series of games about cataloguing fictional wild creatures of the same name. Over the years, several games have been released, and each generation of games – a total of nine, at the time of writing, – introduces (and refines) mechanics, regions and creatures. Much of this information is publicly available on Wikidata. New subproperties of those present in Wikidata were created (but not characterized) that pertain to specific relationships in the Pokémon domain – Wikidata’s properties, by design, are high level and lack the nuance necessary to describe specific domains beyond very simple connections between its individuals (e.g., part of, instance of).

The experiments detailed below show how axiom scoring can be used to determine which axioms could be associated with each property, and how their usage changes with each new generation of games. Table 9 shows the values and thresholds employed, and the application of both ARI and the percentage of selective confirmations ($cf_t$) for axiom scoring are described below. Each generation will be analysed as a different version of the ontology, provided by its own timeframe, and introductions and changes to its properties are documented. As previously established, an acceptation threshold of 0.5 will be used for both strong and weak forms of ARI ($sf_t$ and $wf_t$, respectively) and a window size ($w_s$) of 50.

## 5.1 Section I – ARI and Support

### 5.1.1 Generation I.
Generation I (Gen I) features games with relatively simple mechanics. It introduces one region, one Pokédex (the Pokémon encyclopaedia) and 151 Pokémon. Using the data available on Wikidata, enough information was extracted to ascertain the properties described below. Table 10 shows the names, percentage of selective confirmations, proposed property axioms and ARIs for each.

bordersWith, a property that establishes a connection between two locations (e.g., cities or roads) that share a border, seems a target candidate for S, but the results do not support it (selective confirmations amounted to 1%). Interestingly, the results show there is sufficient positive evidence for T, and adding this axiom to the ontology does not make it inconsistent – although including it would allow for entailing incorrect conclusions about the data. Counterexamples were found for
<table><caption>**Table 10.** Gen I Properties.</caption><thead><tr><th>Property Name</th><th>Axiom</th><th>%Cf</th><th>ARI</th></tr></thead><tbody><tr><td>bordersWith</td><td>T</td><td>51%</td><td>0.87</td></tr><tr><td>hasEvolutionGroup</td><td>AS<br>F</td><td>100%<br>100%</td><td>1</td></tr><tr><td>hasMoveType</td><td>F<br>AS<br>IR</td><td>100%<br>100%<br>100%</td><td>1</td></tr><tr><td>hasPart</td><td>IF<br>AS</td><td>100%<br>100%</td><td>1</td></tr><tr><td>hasPokeDEXEntry</td><td>F<br>IF<br>AS<br>IR</td><td>100%<br>100%<br>100%<br>100%</td><td>1</td></tr><tr><td>introducedIn</td><td>F<br>AS<br>IR</td><td>100%<br>100%<br>100%</td><td>1</td></tr><tr><td>locatedIn</td><td>F<br>AS<br>IR</td><td>100%<br>100%<br>100%</td><td>1</td></tr><tr><td>partOf</td><td>F<br>AS<br>IR</td><td>100%<br>100%<br>100%</td><td>1</td></tr><tr><td>presentIn</td><td>F<br>AS<br>IR</td><td>100%<br>100%<br>100%</td><td>1</td></tr><tr><td>hasValue</td><td>F</td><td>100%</td><td>1</td></tr><tr><td>hasHeight</td><td>F</td><td>99%</td><td>-0.11</td></tr><tr><td>hasWeight</td><td>F</td><td>96%</td><td>-0.28</td></tr><tr><td>hasFacet</td><td>F</td><td>100%</td><td>1</td></tr><tr><td>hasName</td><td>F</td><td>100%</td><td>1</td></tr><tr><td>hasPokeDEXNumber</td><td>F</td><td>100%</td><td>1</td></tr><tr><td>hasColor</td><td>F</td><td>99%</td><td>-0.16</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen I: Generation I.

Explanation for: owl:Thing SubClassOf owl:Nothing

1) '0122 Mr.Mime' 'has Weight' "54.5" In NO other justifications

2) Functional: 'has Weight' In 5 other justifications

3) '0122 Mr.Mime' 'has Weight' "120.2" In NO other justifications

<caption>**Figure 19.** Reasoner's explanation for inconsistency for property hasWeight.</caption>

AS and IR. locatedIn is another property related to the geography of the region. However, since only one region has been introduced as of Gen I, it is classified as *F*, with no contradictions on the data.

With each Pokémon belonging to a single evolution group (described by the hasEvolutionGroup property) but each group having more than one creature, the expected axiom for this property would be *F*, which the results support.

Of the datatype properties, three of them were classified as *F* even though counterexamples were found – since the percentage of selective confirmations was considered sufficient, and the deviations should pertain to possible errors in the data. In this case, we can verify if this was the cause, by using the reasoners provided by Protégé (in this case, HermiT²) and adding said axioms to the ontology and analysing the explanations provided in case inconsistencies are found.

Figure 19 shows the inconsistency explanations for the hasWeight property, in which it is possible to see there are 6 individuals with more than one entry, and the duplicates' values seem to correspond to the same value under different representation systems (metric vs imperial), which suggest that using the same property to represent both may not be adequate.

Figure 20 shows how there is one single individual that contradicts the functionality of the hasHeight property, with the same apparent justification as the hasWeight.
<figure><img src="image_21.png" /><figcaption>**Figure 20.** Reasoner's explanation for inconsistency in property hasHeight.</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>**Figure 21.** Reasoner's explanation for inconsistency in property hasColor.</figcaption></figure>

<table><caption>**Table 11.** Gen II Properties.</caption><thead><tr><th>Property</th><th>Axiom</th><th>%Cf</th><th>ARI</th></tr></thead><tbody><tr><td>bordersWith</td><td>+T</td><td>65%</td><td>0.94</td></tr><tr><td>hasEvolutionGroup</td><td>+F</td><td>100%</td><td>1</td></tr><tr><td>hasMoveType</td><td>+F</td><td>100%</td><td>1</td></tr><tr><td>hasPEntry</td><td>-F</td><td>0%</td><td>-1</td></tr><tr><td>presentIn</td><td>-F</td><td>37%</td><td>-0.93</td></tr><tr><td>alternateDexEntry</td><td>+F</td><td>50%</td><td>0.86</td></tr><tr><td></td><td>+IF</td><td>100%</td><td>1</td></tr><tr><td></td><td>+T</td><td>50%</td><td>0.87</td></tr><tr><td></td><td>+S</td><td>50%</td><td>0.87</td></tr><tr><td>hasGenderRatio</td><td>+F</td><td>100%</td><td>1</td></tr><tr><td></td><td>+AS</td><td>100%</td><td>1</td></tr><tr><td></td><td>+IR</td><td>100%</td><td>1</td></tr><tr><td>hatches</td><td>+T</td><td>25%</td><td>0.66</td></tr><tr><td>hasShape</td><td>N/A</td><td>N/A</td><td>N/A</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen II: Generation II.

Figure 21, on the other hand, shows there are 2 individuals with more than one colour, both belonging to the same evolutionary group. This may be an oversight in the data acquisition from Wikidata, which often mixes the information of all generations.

### 5.1.2 Generation II.
Generation II (Gen II) improves on Gen I by introducing a new region (adjacent to the first one), while still allowing the player to visit the one introduced in Gen I. The Pokédex is expanded to accommodate for the new region: The player now effectively can access not one, but two Pokédexes, one at the national level (with all creatures) and a regional one (with the creatures inhabiting the new region exclusively, which may or may not be new). Therefore, the same creature may now be associated with more than one Pokédex entry, but each entry will be associated to its own numbering system (and potentially, description). Information about the creature's shape is also added, and the number of Pokémon grows from 151 to 251. Additionally, some new game mechanics are included: Creatures can now have one of three genders (female, male, and unknown), and can reproduce within a given group (not necessarily only with members of the same species).

Information about the properties present in Gen II is displayed in Table 11. For the sake of brevity, only changes in axioms are shown. If an axiom is removed, it is preceded by *a* - symbol, and by *a* + otherwise.

hasPotential, which relates a Pokémon to its corresponding Pokédex information, loses the *F* axiom - as the new region introduced a new Pokédex, and therefore a Pokémon may have more than one entry. However, it retains the *IF* axiom, as each entry relates to a single creature. presentIn, a property which describes in which generation a given entity or mechanic is featured, can now point to more than one option and, therefore, is no longer *F*. Of the new properties, alternateDexEntry connects any two entries in different Pokédexes that describe the same creature and therefore should
<figure><img src="image_23.png" /><figcaption>**Figure 22.** Inconsistencies with the use of the hasName property.</figcaption></figure>

<figure><img src="image_24.png" /><figcaption>**Figure 23.** Inconsistencies with the use of the hasColor property.</figcaption></figure>

be classified at least as S. The results show not only this happens, but it also does not contradict the *T*, *F* and IF property axioms.

With the introduction of genders, each species displays one of several possible gender ratios (via the hasGenderRatio property), and as such has been classified as *F*. As no evidence was found against it, it is also classified as AS and IR. Hatches, which relates a creature with its reproductive group, and each reproductive group with the Pokémon in it, has sufficient ARI to be classified as *T*, but not *S*. Finally, the only datatype property added in Gen II, hasShape, does not meet the criteria for any of the property axioms.

When adding the proposed axioms to the object properties in the ontology of Gen II, no inconsistencies are generated. However, the same cannot be said for the datatype properties. According to Figure 22, there are some inconsistencies regarding the use of the hasName property, namely when describing the evolutionary groups. Because new Pokémon were introduced in between generations and added to existing evolutionary groups, some of their names to have undergone changes that have not been corrected.

As seen in Figure 23, in addition and similar to the inconsistencies present in Gen I, there are a few (six) entities with two or more uses of the property hasColor.

### 5.1.3 Generation III.
Generation III (Gen III) introduces two more regions, and no longer features those of Gens I and II. It also introduces two new mechanics: Abilities and contests. Each Pokémon can now have one or two of the 77 possible abilities – some of which are considered ‘signature abilities’, as they are only shown for specific Pokémon or specific evolutionary groups. Furthermore, 135 new Pokémon are added (to a total of 386). The changes and additions to the properties and their axioms are shown in Table 12.

With the introduction of contests, moves now have no longer only a type in battle, but a type that shows in contest (the two are not related). As such, the property hasMoveType is no longer compatible with functionality. Once again, with the introduction of a new region and a new Pokédex, the same Pokémon can have multiple entries – and although these are still alternatives to each other, and therefore symmetrical, there is now more than one possible alternative and the property can no longer be classified as either *F* or *IF*.

Finally, hatches maintains its *T*. Evidence against IR was below the minimum threshold (selective confirmations amounting to 95% of the data) – but as it was already classified as such, the axiom is not added. Following the definition of a signature ability discussed above, the property is properly classified as *F* (each Pokémon/evolutionary group has only one signature ability) and *IF* (each signature ability is used by only one Pokémon/evolutionary group). Since no counterexamples for *IR* and *AS* are found, these axioms are also added to the property. As the percentage of counterexamples found for *F* in hasWeight has once again lowered below the minimum threshold, the property axiom is reinstated.

As with Gen II, when the axioms discovered are added to object properties in the ontology for Gen III, they do not produce inconsistencies. However, with support for hasColor at 98% and hasHeight as 99%, counterexamples are rare but produce some inconsistencies.
<table><caption>**Table 12.** Gen III Properties.</caption><thead><tr><th>Property</th><th>Axiom</th><th>%Cf</th><th>ARI</th></tr></thead><tbody><tr><td>hasMoveType</td><td>-F</td><td>0%</td><td>-1</td></tr><tr><td rowspan="3">alternateDexEntry</td><td>-F</td><td>65%</td><td>-0.76</td></tr><tr><td>-IF</td><td>53%</td><td>-0.9</td></tr><tr><td>-T</td><td>35%</td><td>0.76</td></tr><tr><td rowspan="4">hasSignatureAbility</td><td>+F</td><td>100%</td><td>1</td></tr><tr><td>+IF</td><td>100%</td><td>1</td></tr><tr><td>+AS</td><td>100%</td><td>1</td></tr><tr><td>+IR</td><td>100%</td><td>1</td></tr><tr><td rowspan="4">isSignatureAbilityOf</td><td>+F</td><td>100%</td><td>1</td></tr><tr><td>+IF</td><td>100%</td><td>1</td></tr><tr><td>+AS</td><td>100%</td><td>1</td></tr><tr><td>+IR</td><td>100%</td><td>1</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen III: Generation III.

<table><caption>**Table 13.** Gen IV Properties.</caption><thead><tr><th>Property</th><th>Axiom</th><th>%Cf</th><th>ARI</th></tr></thead><tbody><tr><td>alternateDexEntry</td><td>+T</td><td>47%</td><td>0.85</td></tr><tr><td rowspan="2">hasAlternateForm</td><td>+T</td><td>62%</td><td>0.92</td></tr><tr><td>+S</td><td>27%</td><td>0.69</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen IV: Generation IV.

<table><caption>**Table 14.** Gen VI Properties.</caption><thead><tr><th>Property</th><th>Axiom</th><th>%Cf</th><th>ARI</th></tr></thead><tbody><tr><td rowspan="4">hasMegaEvolution</td><td>+F</td><td>98%</td><td>-0.21</td></tr><tr><td>+IF</td><td>100%</td><td>1</td></tr><tr><td>+AS</td><td>100%</td><td>1</td></tr><tr><td>+IR</td><td>100%</td><td>1</td></tr><tr><td rowspan="4">uses</td><td>+F</td><td>96%</td><td>-0.23</td></tr><tr><td>+IF</td><td>100%</td><td>1</td></tr><tr><td>+AS</td><td>100%</td><td>1</td></tr><tr><td>+IR</td><td>100%</td><td>1</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen VI: Generation VI.

### 5.1.4 Generations IV and V. Generation IV (Gen IV) introduces once again a new region and, with it, comes a new catalogue and new Pokémon (totalling 493). There is also the introduction of 47 new abilities and 113 new moves. Several evolutionary groups from Gen I were expanded with new elements, and moves are now classified not only according to their previously known contest and type categories, but with an additional damage category that is fully independent from its type. It is also in this generation that the games make use of alternate forms of the same Pokémon (including, but not limited to, differences by gender). Generation V (Gen V) raises the total number of Pokémon to 649, and once again introduces a new region while limiting access to the previous ones, but does not introduce any new properties.

Changes and additions to the properties of the ontology for Gen IV are displayed in Table 13:

The inclusion of these axioms in both Gen IV and Gen V ontologies does not cause inconsistencies beyond those already discovered in previous generations.

### 5.1.5 Generation VI. Generation VI (Gen VI) introduces 72 new Pokémon (to a total of 721), 58 new moves and 24 new abilities (to a total of 617 and 188, respectively). It also adds a new battle mechanic, the Mega Evolution, which is a type of alternative form that can be (temporarily) triggered in battle. Like previous generations, Gen VI introduces a new region and a new Pokédex, while also revisiting the region first introduced in Gen III. Table 14 shows the changes in the property axioms in Gen VI, while Table 15 does so for Gen VII:
<table><caption>**Table 15.** Gen VII Properties.</caption><thead><tr><th>Property</th><th>Axiom</th><th>%Cf</th><th>ARI</th></tr></thead><tbody><tr><td rowspan="2">hasRegionalForm</td><td>+F</td><td>97%</td><td>-0.24</td></tr><tr><td>+IF</td><td>98%</td><td>-0.19</td></tr><tr><td>hasSignatureAbility</td><td>-F</td><td>94%</td><td>-0.33</td></tr><tr><td>isSignatureAbilityOf</td><td>-IF</td><td>95%</td><td>-0.32</td></tr><tr><td>uses</td><td>-IF</td><td>71%</td><td>-0.7</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen VII: Generation VII.

<figure><img src="image_25.png" /><figcaption>**Figure 24.** Rare, but valid individuals which do not support *F* for the hasMegaEvolution and uses properties.</figcaption></figure>

<figure><img src="image_26.png" /><figcaption>**Figure 25.** Rare, but valid individuals which do not support the *F* and *IF* for the hasRegionalForm property.</figcaption></figure>

hasMegaEvolution, a relationship between a Pokémon and its Mega Evolution alternate form, is both *F* and *IF* (theoretically meaning that a Pokémon can only have one mega evolution and that evolution belongs to only one Pokémon). The mechanic is triggered by the usage (with the *uses* property) of different battle items, and each of them causes a specific evolution to occur.

When the axioms are added to the Gen VI ontology, they do not produce inconsistencies – contrary to what would be expected when the percentage of confirmations is below 100%. However, upon closer analysis, it is possible to see that there are indeed few, but valid, cases in which a Pokémon can have more than one mega evolution, caused by the application of more than one item. In the dataset for Gen VI, there are two such occurrences, shown in Figure 24. The reasoner fails to flag these as inconsistencies unless the individuals are explicitly stated as being different (which would always be the case under the UNA).

5.1.6 *Generation VII.* Generation VII (Gen VII) sees the introduction of regional forms, as the same creature adapts to different habitats: Effectively another specific type of alternate form. It also increases the number of Pokémon to 802 and introduces a new region and its respective Pokédex. It revisits the region introduced in Gen I, adding new alternate forms to some previously known Pokémon.

With the introduction of new abilities, Gen VII alters how signature abilities work, and a few Pokémon can now have more than one signature ability – as such, hasSignatureAbility can no longer be *F*, and isSignatureAbilityOf can no longer be *IF*. Thankfully, in this case, the number of counterexamples is above the threshold and the axioms are correctly removed. If the axioms are added to the ontology, by UNA they generate some inconsistencies. Figure 25 shows one such case, in which a valid selective confirmation of a creature has more than one known regional variant.

5.1.7 *Generations VIII and IX.* Generation VIII (Gen VIII) introduces two new regions, each with its individual Pokédex. The national one, which catalogues all creatures, now goes up to 890, with 19 new regional forms. The mega evolution mechanic is removed from the games. Generation IX (Gen IX) introduces another region and its respective catalogue, and
<table><caption>**Table 16.** Gen VIII and Gen IX Properties.</caption><thead><tr><td>Property</td><td>Axiom</td><td>%Cf</td><td>ARI</td></tr></thead><tbody><tr><td>hasSignatureAbility</td><td>+F</td><td>96%</td><td>-0.28</td></tr><tr><td>isSignatureAbilityOf</td><td>+IF</td><td>96%</td><td>-0.27</td></tr></tbody></table>

ARI: acceptance/rejection index; Gen VIII: Generation VIII; Gen IX: Generation IX.

<figure><img src="image_27.png" /><figcaption>**Figure 26.** Evolutionary line with more than one signature ability, a counterexample to the *F* of hasSignatureAbility.</figcaption></figure>

103 new creatures (raising the total to 1008). While it introduces the concept of convergent evolution – creatures that fill the same ecological niches also sharing physical similarities – information about this was not present in Wikidata at this time. Finally, this generation introduces a few more regional forms and a new type of alternative form that is not well described in Wikidata.

Because of alterations on how signature abilities are assigned to evolutionary groups between games, Table 16 shows the changes in the associated properties.

Once again, this is a case in which the inconsistencies refer to valid uses of the property, meaning the application of the axioms is in the wrong, as shown in Figure 26:

## 5.2 Section II – Evolving ARI

Experiments show that allowing for some leeway in terms of inconsistency can result in the discovery of errors in the data, such as duplicates, or potential mistakes in modelling that do not account for the possible alternatives. It also shows that not all inconsistencies are caused by said incorrections, and some valid, but outlier information can get flagged as inconsistent.

Evolving ARI depends not on a minimum threshold on the percentage of selective confirmations, $cf_t$, but on the evidence found in the current timeframe and the conclusions obtained on the previous one (which may be a state of total ignorance, if the property was not present).

The following experiments are divided in two different sections, namely:

(1) Applicability and robustness of ARI for axiom scoring. Considering the dataset was obtained from Wikidata, which is often incomplete and sometimes offers incorrect or even contradictory information, we consider the percentage of selective confirmations does not need to equal 100% for the axiom to be accepted. Furthermore, in this case, the analysis is done from a point of complete ignorance, in which no previous versions of any axiom are considered. This should allow for a more informed decision about which thresholds to consider for axiom inclusion in the following experiments.

(2) Analysis of $ARI_e$ over several different timeframes, in which previous versions of the ontology affect the scoring of the new axioms. Using the previously defined thresholds for axiom inclusion and window size, it is possible to measure the effect of more conservative vs progressive approaches to knowledge evolution.

First, we must consider how conservative the approach will be by defining the relative weight of previous knowledge, $w_p$. Then, we must establish a threshold for inclusion or rejection of the hypothesis considering the computed $ARI_e$ ($sf_t$ or $wf_t$, depending on which form was applied). These two values cannot be completely independent of one another, since we are considering only scalar values for the previous state and it is possible for new data to directly contradict said state. As such, if the threshold for inclusion is too high, it may become impossible to change the opinion about an axiom between timeframes.

To avoid this, $sf_t$, $wf_t$ and $w_p$ should be inversely proportional. Since ARI cannot go over 1, $sf_t$ must be below or equal to 0.5 to allow evolution. Using $ARI_e$ also allows for some data in a timeframe to contradict the axiom that is asserted in that period.
<table><caption>**Table 17.** Values and Thresholds Employed in the Experiments.</caption><thead><tr><td>Variable</td><td>Value</td></tr></thead><tbody><tr><td>sf<sub>t</sub>, wf<sub>t</sub></td><td>0.5</td></tr><tr><td>w<sub>p</sub></td><td>0.8/0.7/0.4</td></tr><tr><td>w<sub>s</sub></td><td>50</td></tr></tbody></table>

<figure><img src="image_28.png" /><figcaption>**Figure 27.** Evolution of ARI<sub>e</sub> for hasAlternateDexEntry's *F* with different relative weights.</figcaption></figure>

<figure><img src="image_29.png" /><figcaption>**Figure 28.** Comparison of the decisions made using ARI<sub>e</sub> and ARI+cf<sub>t</sub> for *F* in hasColor.</figcaption></figure>

Make Table 17 shows the thresholds employed for the following experiments – maintaining the acceptance thresholds for the strong and weak forms of ARI (sf<sub>t</sub> and wf<sub>t</sub>, respectively) and a window size (w<sub>s</sub>) of 50 – in which different weights of previous knowledge for ARI<sub>e</sub> (w<sub>p</sub>) will be compared.

ARI<sub>e</sub> depends on the results obtained in previous iterations, so its application can be measured from Gen II onwards. When previous decisions (valued 0 if not present, and 1 for present) about the property axioms are not known, an ARI<sub>e</sub> of 0 is assumed, and the decision for each previous non-existent axiom is scored at 0.5, both to reflect a state of ignorance. The rest of this section presents and discusses the cases in which the decision supported by ARI<sub>e</sub> is different than when using the combination of cf<sub>t</sub> and ARI.

First of all, it is interesting to see the effects of the different weights affect the evolution of ARI<sub>e</sub>. Figure 27 shows how a more conservative approach tends to favour whichever initial decisions were taken, while lower values require less evidence for a decision to be revoked. In this case, the conservative approach would be wrong, as classifying hasAlternateDexEntry as *F* would generate a large number of inconsistencies that would only grow with each generation.

Starting with the datatype properties, which were the ones creating the most inconsistencies since early generations, we can see that without taking cf<sub>t</sub> in consideration, the duplicates are taken in consideration and the decision, across all generations, is not to include the *F* axiom. Figure 28 shows the results for the hasColor property, although they are similar for the others previously mentioned (hasWeight and hasHeight). This effectively means that *F* can no longer be employed to identify errors in data that resulted in the inconsistencies described above.

Differences in the decisions reached ARI, ARI+ cf<sub>t</sub> and ARI<sub>e</sub> are shown by the hatches property, as seen in Figure 29 for *T*.

With the more conservative approach to change provided by ARI<sub>e</sub>, the property is equally classified as *T* for the first two generations, but manages to maintain said status afterwards, despite the gradual decrease in ARI. In this case, adding the axioms to any of the versions of the ontology does not generate any inconsistencies.

In some generations, isSignatureAbilityOf and hasSignatureAbility were shown to have valid counterexamples that were not considered because of the chosen value for cf<sub>t</sub>. Figure 30 shows the effect of the different w<sub>p</sub> in the decision-making process. The results show that employing lower w<sub>p</sub> results consistently in a Functional hasSignatureAbility, which
<figure><img src="image_30.png" /><figcaption>**Figure 29.** Decisions regarding the *T* of the hatches property.</figcaption></figure>

<figure><img src="image_31.png" /><figcaption>**Figure 30.** Comparison of the decisions to include/exclude the *F* axiom for the hasSignatureAbility property.</figcaption></figure>

<figure><img src="image_32.png" /><figcaption>**Figure 31.** Comparison of the decisions to include/exclude the *IF* axiom for the isSignatureAbilityOf property.</figcaption></figure>

<figure><img src="image_33.png" /><figcaption>**Figure 32.** Comparison of the decisions to include/exclude the *F* axiom for the hasRegionalForm property.</figcaption></figure>

is known to produce some inconsistencies from Gen VIII onwards. The opposite happens with heavier $w_p$: By making it harder to change opinion, even though ARI$_e$ gets slightly higher – and in some cases, even positive – values than ARI, the evidence is not sufficient for a change. Figure 31 shows similar trends for IF of isSignatureAbility property.

hasRegionalForm, which was considered *F*, *IF* and *S*, is now instead classified as *T* and *S* for all generations in which it is featured. This happens because ARI$_e$ cannot sustain *F*, as seen in Figure 32.

Since *F* is no longer associated with the property, it can assume the *T* axiom, for which it did have sufficient ARI and ARI$_e$, as Figure 33 shows:

The inclusion of the *T* and *S* axioms to the ontologies of Gens VIII and IX does not produce inconsistencies.
<figure><img src="image_34.png" /><figcaption>**Figure 33.** Comparison of the decisions to include/exclude the *T* axiom for the hasRegionalForm property.</figcaption></figure>

## 5.3 Discussion

Using ARI by itself does not allow for inconsistencies to arise, which in turn means that in spite of its utility for axiom scoring, it does not allow for the very likely possibility of errors and inconsistencies in the data – which is especially relevant when developing an ontology from publicly-available information from sources such as Wikidata. By combining ARI and $cf_t$, we allow for some level of counterexamples to be possible while still accepting an axiom hypothesis. In these cases, only the analysis of the data can show if the decision is correct or incorrect – for example, by investigating any generated inconsistencies to ascertain if they effectively correspond to errors, mistakes, or valid information. Nonetheless, we find the approach helpful for deciding on the inclusion of property axioms on evolving ontologies, and to help an ontology engineer curating the data by adding those axioms and checking any inconsistencies they introduce – aiding the process of identifying potential errors.

The results show that is still possible to counter some errors in data when using $ARI_c$, but not as effectively as the combination of ARI and $cf_t$. This is because, by favouring previous knowledge, $ARI_c$ may have difficulty changing its opinion even in the face of overwhelming evidence. On some occasions, this may be beneficial: The experiments show that, when applied for *T* and *S* – the two property axioms that use the weak-form of ARI because of their inherent missing information – their inclusion is favoured earlier, and is harder to dismiss, with no inconsistencies resulting from it. However, for the properties using the strong form, the effect is diametrically opposite: Axioms are maintained, even though they may introduce an increasing number of inconsistencies over time. Finding a balance between these two outcomes may be possible by tinkering with different values for $sf_t$ and $wf_t$.

# 6 Conclusions

This paper presented an adaptation of the possibilistic approach to axiom scoring to the context of RDF data streams for ontology evolution. The different approaches to possibility and necessity proposed in literature were recontextualized in terms of their bias towards selective confirmations or counterexamples, and the assumptions regarding the openness of the world under which they operate, and proved effective. Some axioms, namely transitivity and symmetry, benefit from a more lenient approach, relying more on selective confirmations than on counterexamples – while the others benefit from stricter acceptance conditions to prevent the proliferation of inconsistencies.

To test the applicability of the solution, it was applied in two distinct scenarios: (1) A first one, in which the property axioms were previously known, and which allowed for the exploration of the effectiveness of the approach for their discovery in a scenario where no incorrect data was present; and (2) a second one, in which the neither the properties nor their axioms were known, and the dataset was obtained from publicly available sources, possibly both incomplete and with errors.

Regarding Experiment I, results show that possibilistic approach is well suited to suggest potential axioms for ontology properties in an instance-guided ontology evolution scenario, achieving conclusions about inclusion/exclusion of axioms from a relatively small sample of individuals (roughly 2.3%), and substantially faster than using traditional information retrieval metrics – making it particularly suitable for quickly learning new axioms from streams of RDF data. This, however, is not achieved without some caveats: The approach is not sufficiently robust to cases in which there are inconsistencies or errors in data, with the strong approach being particularly unable to recover from counterexamples that may reflect said errors. No approach can effectively deal with all potential negative side effects of dealing with an open world and with incomplete knowledge; if ARI is used independently of support, it cannot account for missing or incorrect information.
Additionally, this experiment shows that the size and variety of the individuals in the dataset affect the speed and accuracy of the identification of axioms, suggesting that some finetuning of parameters may be necessary to achieve the best results depending on the application scenario.

Experiment II aimed to overcome the issue with ARI identified in Experiment I in two ways: (1) By combining it with a minimum percentage of selective confirmations and (2) by attributing weight to previous knowledge between timeframes. Axioms that are not 100% supported by ARI may therefore be accepted: This allows for the identification of potential errors in data but may also erroneously suggest discarding valid information. Acknowledging the information provided by previous versions of the ontology when deciding for or against an axiom is helpful in identifying and maintaining property axioms for which positive evidence is inherently harder to find – however, it also allows for the continued integration of incorrect axioms. Here, the combination of support and ARI seems to achieve the better results when it comes to introducing the least amount of inconsistencies over time; the downside being that it may erroneously consider valid, but marginal uses of properties as irrelevant, and therefore the results still need to be analysed on a case-by-case basis. Ultimately, the experimental results show that ARI can be made more resistant to errors in data, benefiting from being combined with other metrics – but more work is needed in this regard to further explore which metrics – and in which way – can be combined in a more systematic fashion that is less dependent of empiric analysis of specific use-cases.

Finally, and although it was out of the scope of the experiments performed for this work, it is also important to consider the possibility of over-characterization of the properties in an ontology. The results show that the current solution will almost always propose one or more axioms for each property – which may or may not make sense, depending on the application context and purpose of the ontology; upper-level ontologies, by definition and application, benefit from excluding superfluous axioms. Even for lower-level ontologies, there is always an argument to be done in favour of simplicity of design, which should not be more complex than necessary to achieve its goals. In the future, the suitability of the suggested axioms should not rely solely on the fact that their addition to the ontology does not introduce inconsistencies, but be motivated too by their capacity to allow for richer inference processes to happen in order to unlock the data's true potential.

## Funding

The authors received the following financial support for the research, authorship, and/or publication of this article: This work has been supported by the European Union under the Next Generation EU, through a grant of the Portuguese Republic's Recovery and Resilience Plan Partnership Agreement under the project PRODUTECH R3. It has received Portuguese National Funds through Portuguese Foundation for Science and Technology under the project UIDP/00760/2020 and the Ph.D scholarship with reference SFRH/BD/147386/2019. Additionally, this work benefited from the informed and pertinent insights by INRIA's mOeX team and the efforts made by Wikidata's Project Pokémon (https://www.wikidata.org/wiki/Wikidata:WikiProject_Pokémon).

## Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

## ORCID iD

Alda Canito https://orcid.org/0000-0002-4338-7239

## References

- Al-Aswadi, F. N., Chan, H. Y., & Gan, K. H. (2020). Automatic ontology construction from text: A review from shallow to deep learning trend. *Artificial Intelligence Review*, 53(6), 3901–3928. https://doi.org/10.1007/s10462-019-09782-9

- Belhoucine, K., & Mourchid, M. (2018). A survey of ontology learning from text. In L. Jain, S. L. Peng, B. Alhadidi, & S. Pal (Eds.), *Proceedings of 12th international conference on advances in semantic processing (SEMAPRO 2018)* (pp. 14–21). Springer.

- Benomrane, S., Sellami, Z., & Ben Ayed, M. (2016a). An ontologist feedback driven ontology evolution with an adaptive multi-agent system. *Advanced Engineering Informatics*, 30(3), 337–353. https://doi.org/10.1016/j.aei.2016.05.002

- Benomrane, S., Sellami, Z., & Ben Ayed, M. (2016b). Evolving ontologies using an adaptive multi-agent system based on ontologist-feedback. 2016 IEEE Tenth International Conference on Research Challenges in Information Science (RCIS) (pp. 1–10). https://doi.org/10.1109/RCIS.2016.7549292

- Bühmann, L., & Lehmann, J. (2012). Universal OWL axiom enrichment for large knowledge bases. *Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)*, 7603 LNAI, 57–71. https://doi.org/10.1007/978-3-642-33876-2_8/

- Buitelaar, P., Cimiano, P., & Magnini, B. (2005). Ontology learning from text: An overview.

- Burek, P., Scherf, N., & Herre, H. (2019). Ontology patterns for the representation of quality changes of cells in time. *Journal of Biomedical Semantics*, 10, Article 16. https://doi.org/10.1186/s13326-019-0206-4
Canito, A., Corchado, J., & Marreiros, G. (2022). A systematic review on time-constrained ontology evolution in predictive maintenance. *Artificial Intelligence Review*, 55(4), 3183–3211. https://doi.org/10.1007/s10462-021-10079-z/

Canito, A., Nobre, A., Neves, J., Corchado, J., & Marreiros, G. (2023). Using sensor data to detect time-constraints in ontology evolution. *Integrated Computer-Aided Engineering*, 30, 169–184. https://doi.org/10.3233/ICA-230703

Cano-Basave, A. E., Osborne, F., & Salatino, A. A. (2016). Ontology forecasting in scientific literature: Semantic concepts prediction based on innovation-adoption priors. In E. Blomqvist, P. Ciancarini, F. Poggi, & F. Vitali (Eds.), *Proceedings of knowledge engineering and knowledge management (EKAW 2016). In: Knowledge engineering and knowledge management* (vol. 10024, pp. 51–67). Springer. https://doi.org/10.1007/978-3-319-49004-5_4

Cardoso, S. D., Pruski, C., & Da Silveira, M. (2018). Supporting biomedical ontology evolution by identifying outdated concepts and the required type of change. *Journal of Biomedical Informatics*, 87(1), 1–11. https://doi.org/10.1016/j.jbi.2018.08.013

Esling, P., & Agon, C. (2012). Time-series data mining. *ACM Computing Surveys*, 45(1), 12. https://doi.org/10.1145/2379776.2379788

Ghidalia, S., Narsis, O. L., Bertaux, A., & Nicolle, C. (2024). Combining machine learning and ontology: A systematic literature review. https://doi.org/10.48550/arXiv.2401.07744

Hamilton, J. D. (1994). *Time series analysis*. Princeton University Press.

How does it work? (OWL Example Wine Agent). (2003). Standford University. Retrieved March 8, 2024, from http://ksl.stanford.edu/projects/wine/explanation.html

Khadir, A. C., Aliane, H., & Guessoum, A. (2021). Ontology learning: Grand tour and challenges. *Computer Science Review*, 39(1). https://doi.org/10.1016/J.COSREV.2020.100339

Kondylakis, H., & Papadakis, N. (2018). EvoRDF: evolving the exploration of ontology evolution. *Knowledge Engineering Review*, 33, e12. https://doi.org/10.1017/S0269888918000140

Malchiodi, D., & Tettamanzi, A. G. B. (2018). Predicting the possibilistic score of OWL axioms through modified support vector clustering. *Proceedings of the ACM Symposium on Applied Computing* (pp. 1984–1991). Association for Computing Machinery. https://doi.org/10.1145/3167132.3167345

Nguyen, T. H. (2021). Mining the semantic Web for OWL axioms. Université Côte d'Azur.

Nguyen, T. H., & Tettamanzi, A. G. B. (2019). An evolutionary approach to class disjointness axiom discovery. *IEEE/WIC/ACM International Conference on Web Intelligence (WI 2019)* (pp. 68–75). https://doi.org/10.1145/3350546.3352502

Ontology Alignment Evaluation Initiative 2023. (2023) Retrieved February 26, 2024, from http://oaei.ontologymatching.org/2023/

Oregon State University. (2014). Plant Ontology. Retrieved March 8, 2024, from https://planteome.org/

Peixoto, R., Cruz, C., & Silva, N. (2015). Semantic HMC: Ontology-described hierarchy maintenance in big data context. In I. Ciuciu, H. Panetto, C. Debruyne, A. Aubry, P. Bollen, R. Valencia-García, A. Mishra, A. Fensel, & F. Ferri (Eds.), *Proceedings of on the move to meaningful internet systems: OTM 2015 workshops. Lecture notes in computer science* (vol. 9416, pp. 492–501). Springer. https://doi.org/10.1007/978-3-319-26138-6_53

Plant, Ontology™ Consortium (2002). The plant Ontology™ consortium and plant ontologies. *Comparative and Functional Genomics*, 3(2), 137–142. https://doi.org/10.1002/cfg.154

Polleres, A., Pernisch, R., Bonifati, A., Dell'Aglio, D., Dobriy, D., Dumbrava, S., Etcheverry, L., Ferranti, N., Hose, K., Jiménez-Ruiz, E., Lissandrini, M., Scherp, A., Tommasini, R., & Wachs, J. (2023). How does knowledge evolve in open knowledge graphs? *Transactions on Graph Data and Knowledge (TGDK)*, 1(1), 1–59. https://doi.org/10.4230/TGDK.1.1.11

Sais, F., Pruski, C., & Da Silveira, M. (2017). Inferring the evolution of ontology axioms from RDF data dynamics. *Proceedings of the 9th Knowledge Capture Conference (K-CAP 2017). Association for Computing Machinery, Inc*. https://doi.org/10.1145/3148011.3154472

Scheffler, I., & Goodman, N. (1972). Selective confirmation and the ravens: A reply to foster. *Journal of Philosophy*, 69(3), 78. https://doi.org/10.2307/2024647

Su, X., Gilman, E., Wetz, P., Riekki, J., Zuo, Y., & Leppänen, T. (2016). Stream reasoning for the internet of things: Challenges and gap analysis. ACM International Conference Proceeding Series, 13–15 June 2016. https://doi.org/10.1145/2912845.2912853

Tettamanzi, A. G. B., Faron-zucker, C., & Gandon, F. (2014). Testing owl axioms against RDF facts: A possibilistic approach. *Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)*, 8876(1), 519–530. https://doi.org/10.1007/978-3-319-13704-9_39

Tettamanzi, A. G. B., Faron-Zucker, C., & Gandon, F. (2017). Possibilistic testing of OWL axioms against RDF data. *International Journal of Approximate Reasoning*, 91(1), 114–130. https://doi.org/10.1016/j.ijar.2017.08.012

Tettamanzi, A. G. B., Zucker, C. F., & Gandon, F. (2015). Dynamically time-capped possibilistic testing of SubClassOf axioms against RDF data to enrich schemas. *Proceedings of the 8th International Conference on Knowledge Capture (K-CAP 2015). Association for Computing Machinery, Inc*. https://doi.org/10.1145/2815833.2815835

Venu, S. H., Mohan, V., Urkalan, K., & Geetha, V. T. (2017). Unsupervised domain ontology learning from text. In R. Prasath & A. Gelbukh (Eds.), *Proceedings of mining intelligence and knowledge exploration (MIKE 2016). Lecture notes in computer science* (vol. 10089, pp. 132–143). Springer. https://doi.org/10.1007/978-3-319-58130-9_13

Wikidata. (n.d.). Retrieved April 8, 2024, from https://www.wikidata.org/wiki/Wikidata:Main_Page