Inductive Reasoning on Few-Shot Knowledge Graphs with Task-Aware Language Models (2025)

Abstract: Knowledge graphs are dynamic structures that continuously evolve as new entities emerge, often accompanied by only a handful of associated triples. Current knowledge graph reasoning methods struggle in these few-shot scenarios due to their reliance on extensive structural information.To address this limitation, we introduce ENGRAM, a novel approach that enables inductive reasoning on few-shot KGs by innovatively enriching the semantics from both textual and structural perspectives. Our key innovation lies in designing a task-aware language model that activates the language model’s in-context learning ability for structured KG tasks, effectively bridging the gap between unstructured natural language and structured tasks. Unlike prior methods that inefficiently employ classification over exhaustive candidate sets, we recast knowledge graph reasoning from a generative perspective, allowing for direct computation of inference results without iterative enumeration. Additionally, we propose a distant neighborhood awareness strategy to enrich the sparse structural features of few-shot entities.Our experimental findings indicate that our method not only achieves state-of-the-art performance in few-shot scenarios. The tunable parameters of our model are approximately 1% of those in previous language model-based methods, and the inference time has been reduced to 1/10 of that required by previous methods.

## introduction in the real world, knowledge is constantly evolving, leading to the iterative development of knowledge graphs (kgs) as structured representations of real-world knowledge. over time, these kgs accumulate a vast amount of new knowledge, resulting in a long-tail phenomenon @xcite . this segment is characterized by the emergence of * corresponding author numerous new entities, each associated with only a limited number of triples @xcite . given this context, enriching the associative information of these low-resource entities becomes crucial, especially considering the challenges posed by scarce data resources which intensify the complexity of reasoning tasks. therefore, the efficient utilization of limited data to perform inductive reasoning on the associative information of such entities is an urgent issue @xcite . current mainstream methods for knowledge graph reasoning (kgr), such as those based on knowledge graph embeddings (kge), effectively capture the associative structures of triples @xcite . however, kge is fundamentally a transductive approach and struggles to represent entities that were not present during training, limiting its capability for inductive reasoning. as a result, graph neural network (gnn)-based methods have been developed. these methods inductively represent emerging entities by aggregating features from their neighboring entities @xcite . nevertheless, in low-resource settings, these gnns often produce ambiguous structural features due to the limited associated data, as depicted in figure 1 @xcite . furthermore, the recent rise in the use of language models (lms) has led to the development of kgr approaches that leverage lms @xcite . these methods extract features from natural language descriptions of triples. however, the accuracy of language model-based methods has not significantly exceeded those based on gnn, and these models also suffer from low training efficiency @xcite . we believe the suboptimal performance of language models in few-shot scenarios can primarily be attributed to: @xcite heterogeneity between structured tasks and natural language: language models typically process unstructured natural language, which significantly differs from the structured knowledge in kgs. consequently, language models face difficulties in directly comprehending structured triples when applied to kgr tasks. some methods employ manually designed prompt templates to transform triple queries into natural language queries @xcite , figure 1 (b.1) illustrates the activation of relevant knowledge in the language model by means of knowledge probing @xcite . (2) inefficiency of classification-based inference pattern: current methods are primarily implemented through triple classification @xcite , such as the link prediction task in figure 1(b. 2). this process requires iterating through all candidate triples, each assessed by the language model to determine the probability of the triple's accuracy. additionally, this approach incurs significant computational redundancy since the variation only occurs in the candidate entities, resulting in inefficient use of computational resources. to bridge the gap between structured tasks and language models, and to enhance inference efficiency, we propose an inductive reasoning method for few-shot kgs using a task-aware language model (engram). first, we introduce a task-guided prompting method that provides the language model with task-specific examples drawn from the few associated triples of few-shot entities. by doing so, we activate the language model's incontext learning abilities @xcite , enabling it to understand and perform structured kg tasks without the need for manual template design. second, moving away from the traditional classification paradigm, we reconceptualize link prediction as a generative task. this shift allows the language model to produce candidate entities directly, significantly reducing computational overhead by eliminating the need to iterate over all possible candidates. we leverage a pre-built contextual representation pool that stores the textual features of observed entities, facilitating simultaneous scoring of all candidate entities. additionally, we propose a strategy that incorporates information from distant but semantically related entities within the kg. by doing so, we construct a more informative context for each entity, which improves feature differentiation and, consequently, reasoning performance. finally, our method integrates both textual and structural information by computing context scores and triple scores, which are then used to jointly train the language model and the gnn components. we compared our method with current state-ofthe-art approaches based on language models and graph structural features on benchmark datasets. our model demonstrates superior computational efficiency, featuring significantly fewer tunable parameters and reducing the computational complexity during the inference phase. further experiments focus on visual analyses of the in-context learning ability of language models and explore the effects of language models (bert, roberta, t5, llama) with different parameter scales. in summary, our contributions are as follows: few-shot reasoning to make more efficient use of few-shot data, a series of meta-learning-based methods have been proposed for modeling fewshot relations and entities. gmatching @xcite was the first to focus on one-shot relations in kgs. metar @xcite , fsrl @xcite , gana @xcite , and metap @xcite utilize intricately designed gnns to learn meta-knowledge with generalization capabilities. gen @xcite was the first to consider few-shot entities within kgs. bayeskgr @xcite estimated the uncertainty in few-shot kgs and used bayesian neural networks to model the uncertainty in the inference process. rawnp @xcite utilized neural processes to model the distribution of limited data. language models for reasoning in recent years, with the rise of language models @xcite @xcite , research has begun to focus on the textual information in kgs @xcite . for example, kg-bert @xcite was the first to use the bert model to achieve link prediction tasks through triple classification. star @xcite introduced a siamese-style textual encoder that uses a shared-parameter model to encode both query and candidate entities separately, enhancing inference efficiency. simkgc @xcite proposed a contrastive learning framework, which improved the quality and efficiency of negative sampling. csprom-kg @xcite introduced a method based on soft prompts, embedding learnable soft prompts into the triplet inputs to reduce the training overhead of language models. tagreal @xcite implemented an automated template construction method, achieving better knowledge probing results and enhancing the method's scalability. definition 1 (knowledge graph with text). a knowledge graph can be defined as @xmath0, r, t ), where: e is the set of entities, r is the set of relations, t ⊆ e × r × e is the set of triples. at the textual level, each entity and relation is not only identified by its graph structure but also enriched with textual descriptions that provide additional semantic information. formally, this can be defined as: x e is an extended set of entities, where each entity @xmath1, x @xmath2, x e e )|@xmath3; x r is an extended set of relations, where each relation @xmath4 to address these issues, we propose a taskguided prompting method that utilizes triples directly as in-context prompts. this innovative approach provides a more intuitive and seamless connection between structured knowledge and natural language. specifically, for each triple (e 1 , r, e 2 ), we construct formalized query-answer pairs, such as (e 1 , r, ?) → e 2 . this reformulation transforms the original triple into a link prediction queryanswer pair, which enables the model to more effectively understand and process structured knowledge graph link prediction tasks. for an emerging entity e ′ , the observed associated triples are {(e i , r i , e ′ ) or (e ′ , r i , e i )} k @xmath5. thus, we can construct query text and answer text y @xmath6, with the in-context prompts for emerging entity e ′ formalized as the string @xmath7; y 1 ; x 2 ; y 2 ; ...; x k ; y k ]. ## conclusion this paper tackles the issue of inductive reasoning within kgs under few-shot scenarios. to tackle this, we introduce an efficient reasoning method that employs a task-aware language model, activating the in-context learning capabilities of the lm and reducing the data discrepancies between structured tasks and lms. additionally, we have reconsidered the problem of inductive reasoning in kgs from a generative perspective. structurally, we have enabled few-shot entities to perceive distant entities, considerably broadening their perceptual domain. experimental results demonstrate significant improvements in both the accuracy and efficiency of inference.