Educational Material to Knowledge Graph Conversion: A Methodology to Enhance Digital Education (2024)

Abstract: This article argues that digital educational content should be structured as knowledge graphs (KGs). Unlike traditional repositories such as Moodle, a KG offers a more flexible representation of the relationships between concepts, facilitating intuitive navigation and discovery of connections. In addition, it integrates effectively with Large Language Models, enhancing personalized explanations, answers, and recommendations. This article studies different proposals based on semantics and knowledge modelling to determine the most appropriate ways to strengthen intelligent educational technologies.

## introduction knowledge graphs (kgs) structure complex information into nodes and relationships, allowing an intuitive and manipulable representation of knowledge. this structure facilitates the integration of information from diverse sources, improves the ability to perform precise semantic searches, and enhances the inference of new knowledge from existing data @xcite . given these capabilities, kgs have shown significant potential across various domains, including education @xcite . in the educational environment, kgs can transform how educational information is organized and accessed. they integrate data from multiple sources, such as textbooks, research articles and online resources, to link key concepts, theories and relevant authors @xcite . in addition, integration with large language models (llms) can enhance this approach, enabling detailed explanations and accurate answers @xcite . this approach facilitates the search for specific information for students and educators and helps identify hidden relationships between different topics, promoting deeper, interdisciplinary learning @xcite . although many kgs have been proposed in the literature, due to their complexity, they are often limited to small environments @xcite . the construction of kgs has traditionally required laborious data extraction and linking processes based on natural language processing (nlp) and data mining techniques @xcite . however, in recent years, llms have revolutionized the field of nlp, demonstrating a remarkable ability to understand and generate natural language and programming. the potential of llms for automatic kg generation is an emerging area of research @xcite . to address the problem of converting educational materials into kgs for improved content structuring, navigation, and personalization with large language models, this paper explores several key areas: ## review of knowledge graphs in education this section discusses three recent studies that review using kgs and ontologies in education. each study addresses different aspects and applications of these technologies, assessing their impact and challenges. the conclusions of each of these studies are then presented, providing a comprehensive view of the current and future state of kgs in education. additionally, we add the article @xcite that proposes a methodology to build kgs in the educational environment. the proposed scheme will be relevant to the proposed method in section 5. @xcite conclude that kgs are transforming education by providing personalized learning experiences and enriched data for curriculum planning. however, they face challenges such as a lack of standardized formats, limited interoperability, incomplete data, and scalability issues. future research is suggested to address these limitations and explore integrating advanced language models and creating multidomain kgs. @xcite highlights the crucial role of ontologies in educational systems, facilitating structured knowledge representation and curriculum management. although there is no single methodology for their construction, researchers combine several methodologies. recent literature review shows an increase in the use of ontologies in education, highlighting their importance and future potential. @xcite reviews the development of ontologies in e-learning, highlighting methodologies such as neon and methontol-ogy, and the roles of domain experts, developers, and end users. it uses metrics such as relationship richness to assess the quality of ontologies. he concludes that a proper understanding and application of these methods and metrics can improve the efficiency and effectiveness of e-learning systems. finally, chen et al. (2018) describes a system called knowedu developed to automatically construct kgs in education using pedagogical and learning assessment data. knowedu uses nlp algorithms to extract meaningful instructional concepts and educational relationships from heterogeneous data. the methods and results of this study provide a solid foundation for the practical implementation of educational kgs. however, this methodology does not allow for an automatic transition from text to kg. ## text-to-knowledge graph conversion models many integrations exist between llms and kgss, but these only cover one of the text-to-knowledge graph process's tasks, as seen in the review @xcite ). an analysis of models that perform the complete task of moving from text to kg is shown below. several common features and differ-ences are observed in these models. the models are commonly evaluated in zero-shot, one-shot, and few-shot scenarios, measuring various datasets' accuracy and semantic relatedness capability. the differences lie in the base llms chosen, the finetuning techniques applied, and the specific architectures used. the results show that, although there are improvements in certain configurations, there is still ample room to optimize the accuracy and efficiency of kg generation. for instance, in the study by @xcite several models are evaluated on the text to owl conversion task in zero-shot, including bert-large @xcite , pubmed-bert @xcite , bart-large @xcite , flan-t5-large (chung et al., 2022), flan-t5-xl (chung et al., 2022), bloom-1b7 @xcite , bloom-3b @xcite , gpt-3 (brown et al., 2020), gpt-3.5 (openai, 2023), llama @xcite and @xcite . these models were tested on the term typing task using different datasets: wordnet @xcite , geonames @xcite , nci (national cancer institute, national institutes of health, 2022), snomedct_us (snomed international, 2023) and medcin (medicomp systems, 2023). the best results were 91.7 for wordnet @xcite , but significantly lower for the other datasets, with scores of 43.3, 16.1, 37.7 and 29.8, respectively, evidencing considerable room for improvement in the models' ability for this task. they were also evaluated in the entity classification task with the geonames @xcite , umls @xcite , and schema.org datasets, showing scores of 67.8, 78.1 and 74.4, again suggesting considerable room for improvement. finally, in the relationship recognition task with the umls @xcite dataset, a result of 49.5 was obtained, reflecting once again the need for improvement. moreover, the same article presents two tuned models: flan-t5-large @xcite and flan-t5-xl @xcite , which show remarkable improvements in several datasets of the evaluated tasks. for example, for the datasets of the first task, the results were improved to 32.8, 43.4 and 51.8. the results improved to 79.3 and 91.7 in the entity classification task, and in the relationship recognition task, 53.1 was achieved. similarly, in the study by @xcite vicuna-13b @xcite and alpaca-lora-13b @xcite are evaluated in zero-shot on the fact extraction task using the f1 metric for different subsets of the wikidata-tekgen @xcite and dbpedia-webnlg @xcite datasets. the best result for the wikidata dataset @xcite ) is 0.38 for vicuna @xcite and 0.28 for alpaca @xcite and for the dbpedia dataset @xcite it is 0.3 for vicuna @xcite and 0.25 for alpaca @xcite . as in the previous case, it is evident that there is much room for improvement. furthermore, in the study by @xcite , a comprehensive evaluation of extended language models (llms) such as @xcite and chatgpt(openai, 2023) in kg construction and reasoning tasks is performed by experiments on eight datasets and four representative tasks: entity and relationship extraction, event extraction, link prediction, and question and answer. the results show that, although gpt-4 achieves an f1 score of 31.03 in relation extraction on duie2.0 @xcite on zero-shot and 41.91 on one-shot, as well as an f1 score of 34.2 on maven @xcite for event extraction on zero-shot, and a hits@1 of 32.0 on fb15k-237 @xcite for link prediction on zero-shot, these results are improbable. the paper by @xcite presents an innovative approach for generating kgs from text in multiple stages. this approach is divided into two main phases: first, the generation of nodes using the pre-trained language model t5-large @xcite and then the construction of edges using the information from the generated nodes. this method seeks to overcome the limitations of traditional graph linearization approaches by breaking the process into manageable and separately optimizable steps. the model was evaluated on three datasets: webnlg 2020 @xcite , tek-gen @xcite and new york times @xcite , obtaining f1 scores of 0.722, 0.707 and 0.918 respectively, demonstrating its effectiveness. however, it highlights the need for further improvement, especially in edge generation, to optimize the system's performance in various applications. finally, in the study by @xcite , embeddings-based methods, such as sifrank @xcite and sifrankplus, which is an exten-sion made by the authors, enhanced with squeeze-bert @xcite , achieved an f1score of 40.38% in keyphrase extraction. in concept weighting, the sbert-based @xcite strategy achieved an accuracy of 13.9% and an f1-score of 20.6% for the top ten ranked concepts, superior results to the benchmark models with which they were purchased. despite these advances, the results highlight the need to improve the accuracy and performance of the techniques to ensure the effective construction of kgs. ## proposed methodology this section presents an innovative methodology for automatically using an llm to generate kgs from educational materials. existing models like bert-large, gpt-4, vicuna-13b, pubmedbert, bart-large, flan-t5, bloom, gpt-3, gpt-3.5, llama, and alpaca-lora-13b have shown progress in converting text to kgs but still have significant limitations, as seen in the previous section. for example, in term typing tasks, scores were 43.3 for geonames, 16.1 for nci, 37.7 for snomedct_us, and 29.8 for medcin, compared to 91.7 for wordnet. in entity classification, the highest scores were 78.1 for umls and 74.4 for schema.org. fact extraction tasks showed vicuna-13b scoring 0.38 and alpaca-lora-13b scoring 0.28 on wikidata-tekgen. these results highlight the need for new strategies to improve model performance in text-to-knowledge graph conversion in general and particularly in education. to address these limitations, we propose a methodology that involves creating an expert model in natural language and kg language. this model is subsequently refined to convert learning materials into kgs, following a learning object structure that offers a guided and comprehensive teaching experience with multimedia educational content. the methodology comprises two phases: continuous pre-training using a large dataset of kgs and specific fine-tuning with didactic materials. during pre-training, a diverse dataset of kgs from sources like wikidata @xcite , dbpedia @xcite , and yago @xcite will be used to train the model with masking and self-supervised learning. this will enhance the model's understanding of semantic relationships and hierarchical structures, improving its ability to generate coherent and accurate graphs. continual pre-training allows the model to become more expert in its domain, enhancing semantic understanding, training on structured data, flexibility, generalization, bias reduction, and leveraging existing resources @xcite . in the fine-tuning phase, diverse educational materials will be gathered, and their corresponding kgs will be created manually or semiautomatically. this process will necessitate defining a kg schema or leveraging an existing one from the literature that aligns with the proposed use case. specifically, the of the ieee computer society (2020) provides a comprehensive schema and vocabulary for metadata that could be particularly useful. alongside this standard, methodologies and schemes described in the studies by @xcite and @xcite will also be considered. although kgs are not used in @xcite , it becomes clear that a small amount of domain-specific data, such as slides and lecture transcripts, can be extremely valuable for building knowledge-based and generative educational chatbots. slides are enriched with semantic annotations, identifying entities such as definitions, quotes, and examples. this enables knowledgebased to provide accurate and relevant responses by mining directly from this structured data. chen et al. (2018) describes a system developed to build educational kgs using pedagogical and learning assessment data automatically. the methods used in this study for extracting instructional concepts and identifying meaningful educational relationships will provide a solid foundation for the proposed kg scheme. integrating these methodologies is expected to improve the system's effectiveness in automatically generating kgs from educational materials. ## conclusion in conclusion, this article argues that structuring digital educational content as kgs rather than traditional repositories provides significant advantages. kgs offer a flexible, navigable representation of concept relationships, enhancing learning personalization and integration with llms. a methodology to automatically generate kgs from educational texts is proposed, promising to transform access to and organization of educational information for more profound, personalized learning.