# Automatic Ontology-Based Model Evolution for Learning Changes in Dynamic Environments
## 5 Evaluation
### Expert-Based Evaluation  
In an expert-based evaluation, the quality of the ontologies is judged on the basis of expert opinion. While this is a subjective evaluation approach, it is frequently considered a good validation process because it relies on the deep knowledge of external experts who can explore the quality of an ontology. In our case, expert-based evaluation is more about assessing the quality of initial and evolved ontology-based models in terms of coverage of the current surrounding environment. We took into account the coverage of concepts to capture the sufficiency of the ontology concepts for representing the runtime changes occurring in the enclosed environment. To this end, an expert was invited that had a good understanding of ontology development and engineering, such as ontology learning, alignment, and merging. He is considered an expert in his field since he has more than 10 years of experience. In this evaluation approach, the initial and evolved ontology-based models, together with a description of the surrounding environment changes, were given to the invited expert, who then applied his knowledge to assess the coverage of the given ontology-based models by checking all their concepts and identifying uncovered concepts to conclude their coverage in response to the emerging changes in the dynamic environment.  
This evaluation consisted of calculating three well-known metrics: the precision, recall, and F-measure. The precision was used to indicate how accurately the concepts identified in an ontology-based model represented the current surrounding environment, and it was the number of correct concepts in the ontology-based model relative to the total number of concepts in the ontology-based model, as shown in Equation (2):  
\[\text{Precision}=Nb\;\text{of correct concepts in the model}/\;\text{Total }Nb\;\text{of concepts in the model}, \tag{2}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Criteria** & **Description** & **Related OOPSI Pitfalls** \\ \hline Consistency & To ensure that the evolved ontology-based models do not contain any inconsistencies (e.g., contradictory or conflicting output results). & P05: Define incorrect inverse relationship. P06: Involve cycles in hierarchy. P07: Merging dissimilar concepts in the same concept. P19: Swapping intersection and union. P24: Using recursive definition. P04: Creating unconnected ontology elements. P11: Missing domain or range in properties. P12: Missing equivalent properties. P13: Inverse relationships not explicitly declared. P02: Creating class synonyms. P03: Creating “is” relationship place of irrelevant or redundant output results). & P03: Creating “rolls”, “rdf:type”, or “owl:sameAs”. P21: Using a miscellaneous concept. \\ \hline \hline \end{tabular}
\end{table}
Table 6: Basic criteria for evolved ontology-based models’ content evaluation.  
The recall was used to measure the environment coverage of the ontology-based model, and it was the number of correct concepts relative to the total number of possible concepts, as shown in Equation (3):  
\[\text{Recall}=Nb\text{ of correct concepts in the model}/\text{Total Nb of possible concepts}, \tag{3}\]  
The F-measure was used to measure the accuracy of the ontology-based model, and it was the harmonic mean that combined both the precision and recall values as shown in Equation (4):  
\[\text{F-measure}=2\times(Precision\times Recall)/(Precision+Recall), \tag{4}\]