Schema Generation for Large Knowledge Graphs Using Large Language Models (2025)

Abstract: Schemas play a vital role in ensuring data quality and supporting usability in the Semantic Web and natural language processing. Traditionally, their creation demands substantial involvement from knowledge engineers and domain experts. Leveraging the impressive capabilities of large language models (LLMs) in tasks like ontology engineering, we explore schema generation using LLMs. To bridge the resource gap, we introduce two datasets: YAGO Schema and Wikidata EntitySchema, along with novel evaluation metrics. The LLM-based pipelines utilize local and global information from knowledge graphs (KGs) to generate schemas in Shape Expressions (ShEx). Experiments demonstrate LLMs’ strong potential in producing high-quality ShEx schemas, paving the way for scalable, automated schema generation for large KGs. Furthermore, our benchmark introduces a new challenge for structured generation, pushing the limits of LLMs on syntactically rich formalisms.

## introduction graphs have emerged as a vital area of research in artificial intelligence and its foundational disciplines, significantly advancing progress across various domains, including knowledge representation and natural language processing @xcite @xcite . this is especially evident with the rise of large language models (llms) @xcite , where graph-based methods enhance their reasoning capabilities for structured knowledge integration, and graphs serve as rich sources of structured and factual information @xcite @xcite . large knowledge graphs (kgs), such as wikidata @xcite and dbpedia @xcite , are compiled from heterogeneous sources, leading to significant quality issues like redundancy, noise, and ambiguity @xcite . beyond data noise, kgs frequently suffer from modeling issues. a survey by wikimedia deutschland on wikidata's ontology issues revealed conceptual ambiguity and inconsistent modeling in wikidata, stemming from diverse contributor perspectives and inadequate guidelines-challenges common to large kgs @xcite . this can manifest, for example, as entities for a company, its service and application being conflated, with predicates incorrectly shared between them. these quality deficits impede effective kg querying, sharing, and reuse. critically, as kgs underpin tasks like pre-training @xcite @xcite , retrieval-augmented generation @xcite @xcite , and post-training @xcite @xcite for llms, ensuring kg quality is essential for maintaining the factual accuracy and reliability of these downstream ai systems @xcite . as crucial resources for quality assurance, kg schemas are generally classified into semantic, validating, and emergent schemas @xcite . in this work, we focus on validating schemas, which are particularly effective in detecting structural and semantic inconsistencies and thereby ensuring data integrity @xcite . these schemas are defined through shapes, where each shape identifies a set of focus nodes within a graph and specifies the constraints and patterns they must adhere to @xcite . to represent such schemas, several dedicated languages have been developed, including w3c standards like shapes constraint language (shacl) @xcite , shape expressions (shex) @xcite , and pg-schema @xcite . however, developing high-quality validation schemas aligned with user needs remains a substantial challenge @xcite . current automatic schema generation via pattern aggregation often inherits noise and errors from the source kgs @xcite . as a result, the primary approach involves manually writing schemas or refining those produced by basic pattern mining-a process that is both time-consuming and expensive, particularly for kgs with potentially millions of classes @xcite . the demonstrated success of llms in areas like ontology engineering @xcite and graph-based reasoning @xcite highlights their proficiency with structured data. specifically, llms show aptitude for processing structured information, reasoning across graph and text modalities, and performing structured generation @xcite . this aptitude strongly suggests their potential for automatically generating kg schemas. building on this, this paper investigates how llms can undertake this automatic schema generation for large kgs, especially where such schemas are missing or incomplete. specifically, we address the research question: how can llms generate high-quality shex shapes for a given kg? we identify key challenges inherent to this task. first, due to the sheer scale of large kgs, pinpointing the essential information required for accurate shex generation by llms is difficult. in addition, even when relevant information is identified, investigating how llms can effectively leverage this structured data for schema generation is required. moreover, the relative novelty and ongoing development of the shex standard means that publicly available shex corpora are scarce, potentially hindering llm familiarity with the required syntax and conventions. to address these challenges, this paper makes the following main contributions: ## problem formulation the resource description framework (rdf) is a standard model for representing data and is widely used in building kgs @xcite . we define a kg as follows: definition 1 (knowledge graph) a kg in rdf is a directed, labeled graph g, defined as a set of triples. each triple (u, p, o) ∈ g consists of a subject u, a predicate p, and an object o. given distinct sets of iris i, blank nodes b, and literals l, the subject @xmath0, the predicate @xmath1, and the object @xmath2, a literal, or a blank node. in particular, we distinguish non-blank subject nodes based on their role in the taxonomy: classes and instances. classes are primarily connected to their superclasses using predicates such as subclassof. instances are linked to their corresponding classes using predicates like instanceof. to validate instances within a class, we can define a validating schema in shex as follows: definition 2 (shape) a shape schema in shex consists of a set of shapes s. each shape is associated with a class c in a kg. a shape @xmath3, ψ) ∈ s comprises a shape label α and a set of constraints ψ ∈ ψ in the form ψ = (p, τ, κ), where p is a predicate, τ is a node constraint, and κ specifies cardinality. predicates used in shape constraints are drawn from the kg. node constraints τ may belong to several categories, including (1) node kind constraints, (2) datatype constraints, (3) values constraints, (4) xml schema string facet constraints and (5) xml schema numeric facet constraints. in this work, we focus on the first three categories. cardinality κ = (n, m) specify the allowable number of occurrences of a predicate-object pair, where @xmath4, with ' * ' indicating an unbounded upper limit. shex examples are provided in appendix b. the task of schema generation can now be defined as: definition 3 (schema generation) given a kg g, and a class @xmath5, the objective is to generate a shape schema s describing the triples involving nodes in the kg. schema generation is a challenging task for both traditional rule-based methods @xcite and ml approaches, due to several factors. large kgs often suffer from quality issues, and pattern extraction may inadvertently propagate these issues into generated shapes. moreover, the lack of benchmarks and high-quality shex ground truths makes training and evaluation of ml-based approaches particularly difficult. dataset yagos wes classes 36 50 constraints sum 678 1,874 mean 18.83 37.48 median 18 35 instances sum 1,227,509 2,127,696 mean 34,097.47 42,553.92 median 1,104 1,564 ## results we further analyze the performance of models under different matching criteria, using deepseek-v3 on the wes dataset as an example ( as for individual criteria, allowing datatype abstraction provides the most significant performance gain, especially in the global setting. this indicates llms can effectively process given datatype information, finding it less challenging than inferring cardinality. allowing approximate subclass matching improved classification scores more for wes compared with yagos, suggesting predicting the precise class list for referenced shapes is harder for wes, while generating related object classes is not. these findings indicate that although llm-generated schemas may benefit from refinement, the required adjustments are generally less intensive or domain-specific than those for existing automated approaches, while still supporting effective validation. figure 2 shows the distribution of results across five categories (correct and four error types) for models and settings on wes and yagos. error types are: (1) missing predicates, (2) incorrect cardinality, (3) incorrect node constraint, and (4) both cardinality and node constraint are incorrect. global settings effectively reduce missing predicates for models on both datasets. missing predicates are more frequent on wes than yagos, likely due to wes's larger set of candidate predicates. however, global settings show a higher rate of cases where both cardinality and node constraint are incorrect, especially on the wes dataset. specifically for cardinality prediction, we compare llm results with rdfshapeinduction using different ml models based on the accuracy of the ## conclusion the persistent challenge of ensuring data quality in large kgs necessitates effective and automated methods for generating validation schemas. this paper explored the application of llms to this task, introducing the first benchmark for shex schema generation from kgs. this benchmark, comprising two datasets and custom metrics, enabled a thorough assessment revealing llms's ability to produce high-