SL i NT : Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion (2025)

Abstract: Link prediction in knowledge graphs (KGs) requires integrating structural information and semantic context to infer missing entities. While large language models (LLMs) offer strong generative reasoning capabilities, their limited exploitation of structural signals often results in *structural sparsity* and *semantic ambiguity*, especially under incomplete or zero-shot settings. To address these challenges, we propose **SLiNT** (**S**tructure-aware **L**anguage model with **I**njection and co**N**trastive **T**raining), a modular framework that injects KG-derived structural context into a frozen LLM backbone with lightweight LoRA-based adaptation for robust link prediction. Specifically, **Structure-Guided Neighborhood Enhancement (SGNE)** retrieves pseudo-neighbors to enrich sparse entities and mitigate missing context; **Dynamic Hard Contrastive Learning (DHCL)** introduces fine-grained supervision by interpolating hard positives and negatives to resolve entity-level ambiguity; and **Gradient-Decoupled Dual Injection (GDDI)** performs token-level structure-aware intervention while preserving the core LLM parameters. Experiments on WN18RR and FB15k-237 show that SLiNT achieves superior or competitive performance compared with both embedding-based and generation-based baselines, demonstrating the effectiveness of structure-aware representation learning for scalable knowledge graph completion.

## introduction knowledge graphs (kgs) encode real-world facts as structured triples (h, r, t), where h and t denote the head and tail entities, respectively, and r is the relation connecting them. as a backbone for structured knowledge representation, kgs empower a variety of downstream applications such as question answering @xcite , recommendation @xcite , and commonsense * *corresponding author. traditional kgc methods such as transe @xcite , distmult @xcite , and rotate @xcite learn low-dimensional embeddings for entities and relations, and rank candidate triples based on geometric scoring functions. while these models perform well in dense regions of the kg, they often underperform on long-tail entities with sparse local neighborhoods. to mitigate this, some extensions incorporate textual features @xcite or graph-aware context @xcite , but still struggle with generalization and semantic discrimination. recent advances in large language models (llms) have introduced a new paradigm for knowledge graph completion (kgc), where pretrained models leverage semantic priors to generate missing entities from textualized queries @xcite @xcite . to improve grounding, several strategies incorporate kg-derived signals into prompts. instruction tuning @xcite encodes relation semantics and output formats into natural language templates, while structural augmentation @xcite @xcite use local subgraphs or structure-aware demonstrations to better align with graph context. despite these strategies having shown promising improvements in generation controllability and kgawareness, persistent limitations emerge, as illustrated in figure 1 , when examining the link prediction query (?, born_in, salzburg): ## related work prior work on knowledge graph completion (kgc) can be broadly categorized into two paradigms: embedding-based models and generation-based models. embedding-based kgc. classical models such as transe @xcite , distmult @xcite , and rotate @xcite encode entities and relations into continuous vector spaces, scoring triples based on distance or semantic compatibility. while efficient and interpretable, these models depend heavily on dense local structures and struggle with long-tail or sparsely connected entities. later extensions incorporate auxiliary textual @xcite or structural @xcite information to improve robustness, but remain limited in handling diverse or ambiguous semantics. ## methodology slint tackles two key challenges in knowledge graph completion (kgc)-structural sparsity and semantic ambiguity-via three modules: structureguided neighborhood enhancement, contrastive learning, and structure-aware injection. as shown in figure 2 , the pipeline starts from pretrained kg embeddings, sequentially applies enhancement and contrastive supervision, then injects structurederived signals into a frozen llm backbone, while fine-tuning only the lightweight lora adapters for efficient adaptation. problem formulation. we formalize the task as link prediction over a knowledge graph @xmath0, r, t ), where t ⊆ e × r × e denotes a set of factual triples. given an incomplete query @xmath1, r, ?) or (?, r, t), the objective is to find the missing entity, either the head h or the tail t, from the entity set e. following dift, we adopt a two-stage formulation. a pretrained kg embedding model m e is first used to rank all candidate entities, producing a top-m list: ## experiments we conduct comprehensive experiments to evaluate the effectiveness of slint on two widely used knowledge graph completion (kgc) benchmarks: fb15k-237 @xcite and wn18rr @xcite . the statistics analysis of these datasets is provided in appendix b. our experiments aim to answer the following research questions: ## conclusion we present slint, a structure-aware generative framework for knowledge graph completion that injects structure-derived evidence from kg embeddings into a frozen llm backbone with lightweight lora-based adaptation. slint integrates three complementary modules: sgne for neighborhood enhancement, dhcl for dynamic contrastive supervision, and gddi for token-level structure injection. experiments on fb15k-237 and wn18rr demonstrate that slint achieves superior or competitive performance, outperforming both embedding-based and generation-based baselines across multiple metrics, while maintaining robustness under structural sparsity and achieving efficient adaptation without full-model fine-tuning.