# An Ontology-Based Expert System for Rice Disease Identification and Control Recommendation
## 5 RiceMan: Ontology and System Evaluation
### Ontology Evaluation  
#### 5.1.1 Ontology Validation  
We carried out this part of the evaluation with five ontology experts, who were ontology engineer experts from NECTEC (National Electronics and Computer Technology Center) and AIT (Asian Institute of Technology) with experiences in modeling and applying ontologies in their work. Four important criteria were considered:  
1. **Appropriateness:** we concentrated on verifying whether each defined class, axiom, and the object property was appropriately modeled based on the domain knowledge and practical usage from rice plant experts' perspectives. Here, the participants investigated both ontologies by using Protege and provided scores for the defined eight sub-criteria, shown in Table 4. That is, RiceDO and TreatO were modeled with 95.2% and 89.4% appropriateness, respectively.
2. **Consistency:** we performed consistency checking in RiceDO and TreatO according to the six sub-criteria proposed in [48]: (1) no duplicated classes; (2) no class cycles in the class hierarchy; (3) no duplicated properties; (4) no property cycles; (5) no invalid ranges; and (6) no invalid domains. As a result, the participants found a few mistakes in some object properties' domains, and we had corrected them. Finally, all participants agreed that the revised ontologies contain no inconsistency according to the mentioned criteria.
3. **Query validation:** to ensure the practicability of applying the ontologies with respect to the defined CQs, we carried out a task-based evaluation, in which we presented for each task the prepared queries corresponding to the defined CQs and their results. The participants verified whether the results were correct, and the queries were written efficiently. Table 5 shows that every query was formulated correctly and efficiently except for Query#4 (i.e., 92%) due to its semantics' incorrectness. Note that we skipped the evaluation of CQ 3 (i.e., finding the abnormalities of any disease) because it was natural to retrieve axioms of abnormalities modeled in RiceDO. Besides,  
Figure 9: Our evaluation design for both ontologies and RiceMan (the evaluation criteria, the methods used by each evaluation and the specific participants are shown).  
we validated that our shortened axioms usage enabled us to write concise queries. Though this aspect was not part of CQs, it was helpful to realize the effectiveness towards writing practical queries.
4. **Ontology satisfaction:** the participants were given a questionnaire to analyze their ontology satisfaction usage. In summary, the participants were satisfied with RiceDO in terms of identifying rice diseases based on either single or multiple abnormalities as well as its convenience in querying by means of the shortened axioms (score 100%). For TreatO, the participants were satisfied with its query facilities for searching of controls (score 88-92%). Moreover, the participants agreed that both ontologies could improve work efficiency (score 100%) and they were willing to use them (score 96%). Overall, it shows the ontologies gain high satisfaction with an average score of 95.5%. For more details, see Table A1 in the Appendix A.