## 4 KGs Evaluation
### Qualitative Evaluation  
Qualitative evaluation is a method that deeply analyzes the internal decision-making process and inference mechanisms of a knowledge graph model through case studies, capturing aspects such as interpretability and transparency, which are difficult to assess through numerical evaluation. Through such evaluations, one can understand how the model processes information and makes decisions and identify the causes of errors. This, in turn, provides important insights into model improvement and optimization strategies.  
For example, in a study that evaluated internal optimization performance, the validity and completeness of the mutation operator were observed to affect the model performance. This study qualitatively analyzed the efficiency trade-offs to explain how each component of the algorithm contributes to the overall performance [169]. In another study, by incorporating a visual attention mechanism, the evaluation focused on which parts of the input sentence the model concentrated on during the entity-relation extraction process, thereby greatly enhancing the intuitiveness of the result interpretation [66]. A study that qualitatively interpreted the phenomenon in which a Sub-Entity Embedding model consistently outperformed existing methods contributed to a better understanding of the subtle performance differences between models [172]. Additionally, research that strengthened the transparency of the model's decision-making process through a balance between text embeddings and internal information provided a qualitative analysis of how the model reached its decisions [173].  
A study that deeply analyzed the use of temporal logical rules and reliability through cases using the ICEWS and GDELT datasets offered qualitative insights into the inference process of the model and the impact of errors [175]. Another study, through a multi-stage inference technique, qualitatively evaluated the interpretability of legal judgment prediction, clearly revealing the decision-making process in practical applications [177]. In the domain of biomedical relationship extraction, research that analyzed the effects of integrating large-scale language model-based text embeddings with domain knowledge on a case-by-case basis provided intuitive insights into the model's internal workings [178]. Furthermore, a study evaluating the readability of community structures and node layouts in graph exploration and visualization through user studies and case analyses qualitatively demonstrated the model's interpretability in applied settings [181].  
In the e-commerce field, the role of a language model that exhibits high relationship labeling performance, even with limited data, was qualitatively analyzed [182]. In a question-answering task, a study qualitatively discussed the strengths and limitations of the system based on the inclusion of SPARQL modifiers [183]. Additionally, research evaluating the results of EHR data integration among medical institutions alongside domain experts [184] and a study analyzing the interpretability of information extraction after generating a large-scale scientific KG [27] have contributed to an in-depth understanding of the model's practical usability. Moreover, studies that qualitatively interpreted the bias of commonsense knowledge models [186], discussed the challenges of classifying sensitive data with domain expert evaluations [187], and assessed the stability of dynamic KG refinement through case studies [188] are important examples of qualitative evaluation. Finally, diverse qualitative analyses have been conducted in various studies on topics such as clinical data integration, semantic relevance evaluation, enhanced path inference for medical diagnosis, AMR parsing, complex entity resolution, fall risk detection, hydraulic engineering, community exploration, cybersecurity, reverse-engineering question generation, and IoT applications.