# OntoTouTra: Tourist Traceability Ontology Based on Big Data Analytics
## 5 Evaluation  
### Evaluation of the Ontology  
In evaluating the ontology, we verified whether the objectives defined in the \"purpose of the ontology\" stage were met and verified whether the ontology was built correctly. We considered the quality criteria proposed by [PERSON] [66]: clarity, coherence, extensibility, minimal coding bias, and minimal ontological commitment, as the evaluative metrics of the ontology. First, we checked the internal consistency of the ontology; we used the HermiT reasoning [67] tool, included in Protege. Once this reasoner was executed, no semantic, infinite loops or partition errors were found. As a second tool, we used OOPS! [68] to detect pitfalls in the ontology, which listed a minor pitfall related to the URI containing the file extension \".owl.\" As a minor suggestion, we skipped this pitfall. Then, we used the GQM approach of the FOCA methodology [69], consisting of the thirteen questions observed in Table 9. The objective of this approach is to verify the domain and application of the ontology. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & **Classifier** & **Algorithm ot Tool** \\ \hline Determine the polarity & Supervised & nltk.sentiment.sentiment.analyzer \\ Grouping by ratings & Not supervised & K-means \\ Detection of services & Supervised & Named entity recognition (NER) with SpaCy \\ Detection of tourist experiences & Supervised & NER with SpaCy \\ Detection of POIs & Supervised & NER with SpaCy \\ Detection of language & Supervised & nltk.stem \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Tourist review categories. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Goal** & **Question** & **Metric** & **Note** & **Question Grade** & **Goal Grade** \\ \hline \multirow{4}{*}{1. Check if the ontology compiles with substitutes} & Q1. Were the competency questions defined? & Completeness & 13 KPIs as CQ & 100 & \\  & Q2. Were the competency questions & Completeness & 13 KPIs answered & 100 & 83.3 \\  & Q3. Did the ontology reuse other ontologies? & Adaptability & Open link data with GeNames and Time Ontology & 50 & \\ \hline \multirow{4}{*}{2. Check if the ontology compiles with ontological commitments} & Q4. Did the ontology impose a minimal ontological commitment? & Conciseness & Ontology uses abstractions to define concepts & 75 & \\  & Q5. Did the ontology impose a maximum ontological commitment? & Conciseness & Ontology does not use many primitive concepts & - & 75 \\  & Q5. Are the ontology properties coherent with the domain? & Consistency & Checked by HermIT reasoning (Protegg plugin) & 75 & \\ \hline \multirow{4}{*}{3. Check if the ontology compiles with intelligent reasoning} & Q7. Are there contradictory axioms? & Consistency & Checked by HermIT reasoning (Proteg plugin) & 100 & \\  & Q8. Are there redundant axioms? & Conciseness & Checked by HermIT reasoning (Proteg plugin) & 100 & 100 \\ \cline{1-1} \cline{2-6}  & Q9. Did the reasoner bring modeling errors? & Computational efficiency & 1 minor error; Checked by OOPS! & 75 & \\ \cline{1-1} \cline{2-6}  & Q10. Did the reasoner perform quickly? & Computational efficiency & Depending on Protegé capacity (we ran without the reviews’ individuals: 17.197 ms) & 75 & \\ \hline \multirow{4}{*}{5. Check if the ontology compiles with human expression} & Q1. Is the documentation consistent with modeling? & Clarity & Documentation generated by Protegé & 100 & \\ \cline{1-1} \cline{2-6}  & Q12. Were the concepts well written? & Clarity & We used the ontology annotations (rdkscomment) & 100 & \\ \cline{1-1} \cline{2-6}  & Q13. Are there annotations in the ontology that show the definitions of the concepts? & Clarity & We used the ontology annotations (rdkscomment) & 100 & \\ \cline{1-1} \cline{2-6}  & & & & \\ \hline \end{tabular}
\end{table} 
Table 9: Applying the goal-question-metric approach from the FOCA methodology on the TTS ontology domain.