Taxonomy-Driven Knowledge Graph Construction for Domain-Specific Scientific Applications (2025)

Abstract: We present a taxonomy-driven framework for constructing domain-specific knowledge graphs (KGs) that integrates structured taxonomies, Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). Although we focus on climate science to illustrate its effectiveness, our approach can potentially be adapted for other specialized domains. Existing methods often neglect curated taxonomies—hierarchies of verified entities and relationships—and LLMs frequently struggle to extract KGs in specialized domains. Our approach addresses these gaps by anchoring extraction to expert-curated taxonomies, aligning entities and relations with domain semantics, and validating LLM outputs using RAG against the domain taxonomy. Through a climate science case study using our annotated dataset of 25 publications (1,705 entity-publication links, 3,618 expert-validated relationships), we demonstrate that taxonomy-guided LLM prompting combined with RAG-based validation reduces hallucinations by 23.3% while improving F1 scores by 13.9% compared to baselines without the proposed techniques. Our contributions include: 1) a generalizable methodology for taxonomy-aligned KG construction; 2) a reproducible annotation pipeline, 3) the first benchmark dataset for climate science information retrieval; and 4) empirical insights into combining structured taxonomies with LLMs for specialized domains. The dataset, including expert annotations and taxonomy-aligned outputs, is publicly available at https://github.com/Jo-Pan/ClimateIE , and the accompanying framework can be accessed at https://github.com/Jo-Pan/TaxoDrivenKG .

## introduction effective management and utilization of structured knowledge is a core challenge in domain-specific research. while scientific publications across fields, from materials science to epidemiology, routinely describe critical relationships between models, observational datasets, and analytical findings, these connections are rarely formalized or linked to standardized data sources @xcite @xcite . for instance, climate science papers might detail how green house gas emission affects the occurrence of wildfires @xcite , while chemistry studies could analyze battery chemistry performance under different extreme conditions @xcite . yet in both cases, these insights remain trapped in unstructured text, inaccessible to computational analysis. this lack of systematization impedes cross-study knowledge integration, slowing discovery and limiting reproducibility. knowledge graphs (kgs) address this gap by structuring entities and relationships into semantically interconnected frameworks, enabling querying, automated reasoning, and cross-domain interoperability @xcite . although kgs have advanced research in domains like material science @xcite and geospatial sciences @xcite , constructing them in specialized fields faces two main challenges. first, existing methods overlook domain taxonomies, which are curated hierarchies of verified entities and relationships. instead, they build kgs from scratch via llms. @xcite . while flexible, this forfeits the semantic rigor and community consensus embedded in taxonomies, leading to inconsistent representations. second, despite llms' proficiency in generalpurpose information extraction @xcite , they struggle in specialized domains: hallucinating entities, misclassifying relationships, and overlooking tail-domain concepts absent from their training data @xcite . for example, in climate science, models frequently conflate teleconnections (large-scale climate linkages) with generic correlations or fail to recognize emerging terms like 'arctic amplification'. these errors compromise kg reliability for downstream tasks. a critical bottleneck in kg construction lies in accurate named entity recognition (ner) for specialized domains. state-of-the-art generalist models like gliner @xcite , which achieve competitive performance on broadcoverage benchmarks (f1: 0.478), falter in domainspecific settings-scoring only 0.339 f1 on climate science texts. this performance gap stems from two interrelated issues: 1) domain-specific terminology-such as teleconnections, oceanic rossby waves, and cmip6 emission scenarios-occupies the "long tail" of knowledge underrepresented in llm training corpora @xcite , and 2) llms lack mechanisms to disambiguate domainrelevant entities (e.g., "water" as a model variable in hydrological studies) from semantically similar generic terms (e.g., generic mentions of "water" in non-technical contexts). recent advances in weak supervision @xcite have shown promise in augmenting scarce annotations by leveraging heuristic rules, knowledge bases, or llmgenerated pseudo labels, offering a viable path to improve domain-specific entity recognition at scale. consequently, llms either omit critical concepts or misclassify them, propagating errors into downstream kg components. to address these challenges, we propose a framework that synergizes domain taxonomies, constrained llm extraction, and iterative validation, demonstrated through climate science kg construction. our approach comprises three key components: 1) taxonomy-driven kg construction: extraction is anchored to expert-curated taxonomies (e.g., mesh in biomedicine, nasa's gcmd @xcite in climate science). by integrating rag with llms, we ensure extracted entities (e.g., cmip6 experiments) and relationships (e.g., enso influences drought) align with the taxonomy's hierarchical structure, preserving semantic consistency. 2) constrained entity and relation typing: to reduce hallucinations, we restrict the types of named entities (nes) and relations that llms can extract. this prevents irrelevant entity types, such as person names, from being included. few-shot learning is employed to adapt the model to domain tasks, improving performance. 3) ragbased output verification: unlike approaches like graphrag @xcite , which directly use model outputs for kg construction, we verify outputs using rag against the domain taxonomy. this prevents the introduction of wrong entities and re-lations into the graph. our work advances domain-specific kg construction through the following contributions: while demonstrated in climate science, our framework provides a potential blueprint for constructing kgs in any domain with structured taxonomies (e.g., space domain awareness taxonomy). by anchoring extraction to expert-curated hierarchies, we ensure semantic consistency while enabling sustainable updates. ## method overview we propose a generalizable framework for constructing domain-specific kgs that harmonizes structured taxonomies with unstructured text extraction. while demonstrated through climate science, a domain with complex terminology and figure 1 illustrates the proposed framework for kg construction from scientific publications. we start with a taxonomy, which provides a hierarchical classification of domain-specific named entities but lacks explicit relationships beyond hierarchical structures such as subclass relations. to enrich this taxonomy, we incorporate a broader set of relations that define interactions between entities. these relations are automatically derived from research publications, but are constrained by our rag to predefined types of relations and entities within the taxonomy, ensuring consistency and mitigating hallucinations. the taxonomy serves as the structural foundation of the kg, anchoring entity organization, while the extracted relations add depth by capturing meaningful interactions between entities. ## stage 1: taxonomy integration we propose a 3-step framework to transform domain taxonomies into adaptive backbones for kg construction, applicable to scientific fields requiring structured yet evolving knowledge representation. using climate science as a case study, the process involves: aggregating domain-specific taxonomies, enhancing node definitions, and indexing for semantic alignment. ## stage 3: dynamic kg assembly & maintenance our framework constructs domain-specific kgs that balance taxonomic stability with adaptability. the resulting kg (e.g., climatepubkg for climate science) integrates entities from domain taxonomies (e.g., gcmd+) and scholarly publications into a unified graph database (e.g., neo4j). each relationship inherits provenance metadata-including paper references, cited text snippets, and contextual mentions-enabling evidence-based queries. for instance, in climate science, a measuredat relationship between enso signals and an oceanic location links to the source publication's methodology section. we demonstrate through a climate science case study: processing 300 papers from semantic scholar established 21k validated entitypublication links (e.g., connecting cmip3 models to teleconnection studies). automated pipelines continuously ingest new publications, expanding coverage while enforcing taxonomic alignment. to balance comprehensiveness with reliability, unlinked entities (e.g., emerging terms like "subsurface salinity fronts") undergo systematic monitoring. 1) frequency tracking: entities surpassing occurrence thresholds are flagged. 2) expert validation: domain specialists assess candidates for taxonomy inclusion. 3) taxonomy extension: ap-proved entities are added with unique identifiers. this process filters transient concepts while integrating validated knowledge. the kg architecture supports dual roles: a historical repository and a live research tool. in climate science, feedback loops between experts and extraction models enable real-time hypothesis testing (e.g., validating new teleconnection patterns against historical data). by grounding kgs in taxonomies while accommodating domain evolution, our framework achieves precision at scale-critical for fields like climate science where terminology and relationships evolve rapidly. the methodology generalizes to other domains through configurable taxonomic constraints and validation rules. ## domain-specific annotation pipeline we demonstrate our framework's practicality through a climate science annotation pipeline, validated by 4 domain experts. at each step, the consistency of the annotated entities and relationships was verified, and discrepancies were resolved collaboratively. using the inception annotation tool, @xcite we annotated 25 publications from semantic scholar, covering a wide range of climate science topics, including atmospheric processes, ocean dynamics, and climate modeling. this yielded 13,773 entity mentions (10,174 linked to gcmd+) and 3,618 validated relationships. frequent categories include variable (3,953 mentions), location (2,767), and (climate) model @xcite , as detailed in appendix a.5. by recycling step outputs as inputs (e.g., ner results inform linking), we reduced annotation effort. annotation guidelines and further details on the annotation process can be found in appendix a.9. ## experiments the experiments aim to evaluate the proposed framework's effectiveness and investigate the contributions of its key components, including fewshot learning, rag, backbone models, and relationship extraction. the evaluation is conducted on three tasks: ner, el, and re. ## results and discussion our proposed framework includes all components including 10-shot, prerag, postrag and relationship extraction. experiments yield three key findings. first, taxonomy constraints with llms significantly improves climate science information extraction. second, retrieval augmentation and few-shot learning effectively reduce hallucinations. third, relationship extraction introduces precisionrecall trade-offs requiring careful balancing. ## conclusion in this work, we presented a taxonomy-driven framework for domain-specific kg construction using llms and rag. our approach addresses the challenges of extracting and organizing domainspecific knowledge from unstructured scientific literature. by grounding the kg construction process in a taxonomy (nasa's gcmd), we ensured semantic consistency and reduced hallucinations commonly associated with llms. our experiments demonstrated the effectiveness of integrating rag with llms for kg construction, particularly in improving precision and reducing false positives in entity recognition and relationship extraction. the use of few-shot learning further enhanced the model's ability to adapt to the climate science domain, even with minimal training examples. additionally, our curated dataset and annotation pipeline provide a valuable resource for future research in climate science information extraction. while demonstrated in climate science, our framework provides a blueprint for any domain with structured taxonomies. by converting unstructured text into structured, machine-readable knowledge representation, this work enables large-scale organization of specialized scientific information.