LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs (2024)

Abstract: Knowledge Graph (KG) inductive reasoning, which aims to infer missing facts from new KGs that are not seen during training, has been widely adopted in various applications. One critical challenge of KG inductive reasoning is handling low-resource scenarios with scarcity in both textual and structural aspects. In this paper, we attempt to address this challenge with Large Language Models (LLMs). Particularly, we utilize the state-of-the-art LLMs to generate a graph-structural prompt to enhance the pre-trained Graph Neural Networks (GNNs), which brings us new methodological insights into the KG inductive reasoning methods, as well as high generalizability in practice. On the methodological side, we introduce a novel pretraining and prompting framework PROLINK, designed for low-resource inductive reasoning across arbitrary KGs without requiring additional training. On the practical side, we experimentally evaluate our approach on 36 low-resource KG datasets and find that PROLINK outperforms previous methods in three-shot, one-shot, and zero-shot reasoning tasks, exhibiting average performance improvements by 20%, 45%, and 147%, respectively. Furthermore, PROLINK demonstrates strong robustness for various LLM promptings as well as full-shot scenarios. Our source code is available on https://github.com/ KyneWang/ProLINK.

heading: 1 Introduction
text: Knowledge Graph (KG) Reasoning, also known as KG Link Prediction, aims at inferring new facts from existing KGs in the triple format (head entity, relation, tail entity)(Wang et al., 2021d,b; Li et al., 2023). This technique has been widely studied and applied in various domains, including information retrieval, e-commerce recommendations, drug discovery, and financial prediction (Deng et al., 2019;
* The Corresponding Author
Zeng et al., 2020; Bonner et al., 2022; Ge et al., 2023). For example, the query (q1) in Figure 1 consists of ‘Entity 3’ and a relation type ‘occupation’. Its expected answer for KG reasoning is ‘Entity 6’.
The dynamic nature of real-world KGs fosters recent research interest in inductive KG reasoning—inferring from new KGs which have entities/relations unseen during training (Sadeghian et al., 2019; Teru et al., 2020). In contrast to earlier methods that learn graph-specific embeddings for the training KG (Bordes et al., 2013; Yang et al., 2015; Sun et al., 2019), recent inductive reasoning models are trained to address the query (q1) in the training KG, enabling them to answer the query (q2) in a fully new KG (Lee et al., 2023; Galkin et al., 2023). As illustrated in Figure 1, this transferability stems from the shared interaction patterns in relation graphs, which can be captured by Graph Neural Networks (GNNs) (Kipf and Welling, 2017; Liao et al., 2022; Zhu et al., 2022; Liu et al., 2023b) without using entity/relation textual information.
One critical challenge of KG inductive reasoning is handling low-resource scenarios marked by scarcity of both textual and structural information.
3742
For the query (q3) in Figure 1, the inference KG lacks textual context for entities and has no support triple for the ‘co-worked’ relation. Such lowresource scenarios frequently occur in specialized or emerging domains due to long-tail distribution and new relation additions (Pei et al., 2023; Wu et al., 2023), hindering the wide adoption of KG inductive reasoning. Text-based methods, even the current powerful Large Language Models (LLMs) (Touvron et al., 2023; OpenAI, 2023), are constrained by limited textual data and complicated graph structures, while graph-based methods struggle with few-shot relation types that lack enough interaction edges in the relation graph1.
Human reasoning, however, may address the above low-resource queries without expert knowledge or prior learning. Simply leveraging limited relation semantics to understand graph structures, humans can successfully deduce that the ‘coworked’ relation should connect two persons, suggesting the answer entity can be a head entity of the ‘job’ relation and near the query entity ‘Entity 4’. Inspired by the human reasoning process which mostly relies on relation semantics, we pose a question: Can LLMs be used to emulate human reasoning for relation semantic understanding, thereby enhancing the GNN-based inductive reasoning?
This work gives a positive answer to the above question and presents an approach that leverages LLM’s basic language power to improve KG inductive reasoning in any low-resource scenarios without additional model training for new KGs. This ability is crucial for elevating the generalizability of AI technologies to handle data dynamics in the real world. Specifically, we propose a novel pretraining and Prompting framework PROLINK for Low-resource INductive reasoning across arbitrary KGs. First, PROLINK pretrains a GNNbased KG reasoner with novel techniques enhancing few-shot prediction performance. Then, given a new inference KG with sparse relation types, PROLINK employs a pre-trained LLM to construct a prompt graph through concise relation descriptions (a dozen words per relation). The prompt graph is calibrated to eliminate noise and subsequently injected into the relation graph of the KG to improve the performance of the GNN reasoner. In summary, this work has four novel contributions: • We introduce a new KG reasoning problem, i.e.,
low-resource inductive reasoning on KGs with
1We verified this challenge in preliminaries in Section 2.3.
arbitrary entity and relation vocabularies, which generalizes most KG link prediction tasks.
• To the best of our knowledge, this is the first work leveraging LLMs as the graph prompter for inductive KG reasoning, potentially inspiring further innovation in the research community.
• We design a unique pretraining and prompting framework PROLINK, containing several novel techniques for low-resource inductive reasoning and prompt graph generation.
• We construct 36 low-resource inductive datasets from real-world KGs, in which PROLINK outperforms previous state-of-the-art methods in both few-shot and zero-shot reasoning tasks.

heading: 2 Background

heading: 2.1 Notations and Definitions
text: Let G = {E ,R, T } denote a Knowledge Graph (KG), where E ,R are the sets of entities and relations. T = {(eh, r, et)|eh, et ∈ E , r ∈ R} is the set of factual triples of the KG, where eh, r and et are called a triple’s head entity, relation and tail entity, respectively. Given a query (eq, rq) containing a query entity eq ∈ E and a query relation rq ∈ R, the KG reasoning task aims to identify the correct entity ea ∈ E , such that the triple (eq, rq, ea) or (ea, rq, eq) is a valid factual triple of G. KG Inductive Reasoning: Given a model trained on a knowledge graph Gtr = {Etr,Rtr, Ttr}, the KG reasoning task in the fully inductive scenario evaluates the trained model on a new inference graph Ginf = {Einf,Rinf, Tinf}, in which all entities and relations are different from those in Gtr, i.e., Einf ∩ Etr = ∅ and Rinf ∩Rtr = ∅. K-shot Inductive Reasoning: In the inductive scenario with Ginf = {Einf,Rinf, Tinf}, given a new relation type rq /∈ Rinf and support triples Trq = {(eh, rq, et)|eh, et ∈ Einf}(|Trq | = K), the K-shot reasoning task is to predict over a query set {(eq, rq)|eq ∈ Einf} with the augmented KG Grqinf = {Einf,Rinf ∪ {rq}, Tinf ∪ Trq}.
In this work, we focus on the low-resource challenge of KG inductive reasoning, inferring not merely unseen but also few-shot relation types. Existing methods for few-shot link prediction (Chen et al., 2019; Zhang et al., 2020; Huang et al., 2022; Wu et al., 2023) either only work on the training graph structure or have to use graph-specific textual or ontological information to train, therefore cannot achieve inductive reasoning on arbitrary KGs. Other related studies also have difficulty accom-
plishing this low-resource task, including entitylevel inductive reasoning (Sadeghian et al., 2019; Teru et al., 2020; Zhu et al., 2021; Zhang and Yao, 2022) and text-based inductive reasoning (Daza et al., 2021; Wang et al., 2021e; Markowitz et al., 2022; Gesese et al., 2022). Detailed related work is introduced in the Appendix E.

heading: 2.2 Baseline: ULTRA
text: We first introduce ULTRA (Galkin et al., 2023), the state-of-the-art GNN-based model for inductive reasoning on entirely new KGs. Its core idea is leveraging the ‘invariance’ of the KG relational structure. With the nature of the triple form, there are four basic interaction types when two relations are connected, i.e., tail-to-head (t2h), head-to-head (h2h), head-to-tail (h2t), and tail-to-tail (t2t). As shown in Figure 1, different KGs may have similar patterns in the relation interactions. Thereby, the pre-trained embeddings of four interaction types can be universally shared across KGs to parameterize any unseen relations.
Given an inference KG G = {E ,R, T }, ULTRA constructs a relation graph Gr = {R,Rfund , Tr} from the original triple data T , in which each node is a relation type and edges have four interaction types Rfund . After adding inverse relations into G (Zhu et al., 2021; Zhang and Yao, 2022), Gr would contain 2|R| nodes. Then, ULTRA employs a graph neural network GNN r(·) over the relation graph Gr , and obtains the relative representation of each relation conditioned on a query, which then can be used by any off-the-shelf entity-level GNNbased models GNN e(·) for KG reasoning (Zhu et al., 2021; Zhang and Yao, 2022; Wang et al., 2023). Specifically, given a query q = (eq, rq), the score p(eq, rq, et) of one candidate entity et is calculated as follows:
Rq = GNN r(θr, rq,Gr ), (1) Eq = GNN e(θe, eq,Rq,G), (2) p(eq, rq, et) = MLP(Eq[et]). (3)
where θr, θe denote the parameters of two GNN modules, and eq, rq are initialized embedding vectors of eq and rq2. Because ULTRA does not require any input features of entities or relations nor learn graph-specific entity or relation embeddings, it enables inductive reasoning across arbitrary KGs.
2In the implementation, they are initialized as all-one vectors whereas other nodes in the graph are initialized with zeros, which is verified generalizing better to unseen graphs.

heading: 2.3 Preliminaries: Low-resource Challenge
text: To explore the low-resource challenge, we verify the performance of the GNN-based ULTRA (Galkin et al., 2023) and the LLM-based Llama2 (Touvron et al., 2023) in zero-shot inductive reasoning. On three real-world KG benchmarks, the zero-shot condition is created by removing all rqinvolved triples in Tinf for each query relation rq. The preliminary results of the two pre-trained models are shown in Figure 2.
In Figure 2(a), we observe that the zero-shot performance of ULTRA(3g) drops sharply compared to its original version. It indicates the inductive ability of ULTRA highly relies on sufficient support triples in the inference graph. Surprisingly, ULTRA obtains better performance after adding the complete relation graph built on the original Ginf, which motivates us to enhance the relation graph in low-resource scenarios. Following recent work (Ye et al., 2023), we further evaluate the zeroshot graph reasoning performance of Llama2-7B by converting queries and KG subgraphs into textual questions3. Due to the context window limitations of Llama2, we select hundreds of queries whose answers are in the 2-hop neighborhood subgraph of the query entity. Even though, the results are not promising. In Figure 2(b), aside from issues related to excessive length or incorrect formatting, only dozens of requests obtain standard outputs and even fewer include the correct answer in the top ten entities outputted (Hits@10).
In summary, the above preliminary results expose the challenges faced by existing methods in low-resource scenarios for KG inductive reasoning, highlighting the necessity for innovative solutions to more effectively utilize sparse data.

heading: 3 Methodology
text: This work presents a novel pre-training and prompting framework, PROLINK, which employs pretrained language models (LLMs) and graph neural networks (GNNs) to handle semantic and struc-
3For a fair comparison, both our methods and here use only short relation context instead of entity context.
tural information, respectively. As illustrated in Figure 3, for a few-shot query relation in a new inferring KG, the pre-trained GNN Reasoner (Section 3.1) infers from KG subgraphs without model finetuning, guided by a relation-specific prompt graph. The prompt graph is constructed by a frozen LLM Prompter (Section 3.2) from relation semantics, and then calibrated by Prompt Calibrator (Section 3.3) to mitigate noise.

heading: 3.1 GNN-based Reasoner
text: Our GNN-based reasoner follows the basic framework of ULTRA (Galkin et al., 2023) in Section 2.2. Given a G = {E ,R, T }, we first construct the relation graph Gr = {R,Rfund , Tr}, which is efficiently obtained from the original graph G with sparse matrix multiplications. After that, given a query q = (eq, rq), we generate relative relation/entity representations Rq/Eq from the graph structure of Gr and G, respectively. Then, calculating the triple score p(eq, rq, et) of any candidate entity et follows Equation 3. Specifically, the GNN architecture follows NBFNet (Zhu et al., 2021) with a non-parametric DistMult (Yang et al., 2015) message function and sum aggregation. The relation encoder GNN r(·) utilizes randomly initialized edge embeddings for Rfund . In contrast, GNN e(·) initializes the embeddings of edge types using the relative relation embeddings Rq. We suggest consulting the ULTRA paper (Galkin et al., 2023) for further details. To improve the pre-training performance on low-resource inductive reasoning, we propose two enhancing techniques:
Role-aware Relation Encoding: The embeddings generated by GNN r(·) for relative relations are currently missing vital information: the specific role that each relation type assumes in the reasoning process of a query. We delineate three
unique roles: query relation, inverse query relation, and other relation. As a relation may serve different roles across various queries, its embedding vector should reflect the nuances of its designated role. To imbue this role-aware capability, we introduce trainable role embeddings Ro ∈ R[3×d] to augment each relation embedding via a two-layer MLP, formulated as: r̂ = δ ( W2δ ( W1(Rq[r] : Ro[roleq(r)]) ))
, where Rq[r] is the relative relation vector of r, concatenated with its specific role vector as determined by roleq(·). The enhanced relation embeddings R̂q are then utilized in Equation 2 replacing Rq. This component requires far fewer parameters than the original ULTRA. Therefore, efficiency issues can usually be ignored.
Low-resource Pretraining Objective: The original ULTRA is trained by minimizing the binary cross entropy loss over positive and negative triples. Due to the sufficient support triples in the training KG, the GNN reasoner would prioritize reasoning patterns related to these triples, which would not present in low-resource scenarios. To prompt GNNs to derive insights from a broader range of relations, we devise an extra pre-training task that operates on ’pseudo’ low-resource KGs. Specifically, for queries whose relation is rq, we construct a specific KG Grqtr , where support triples containing rq or its inverse relation are randomly masked, using a hyperparameter γ to determine the masking proportion. The total loss pretraining on the original KG and masked KGs is calculated as follows:
LG = − log pG(eq, rq, ea)− 1 n
n∑
i=1
log(1− pG(eq, rq, ei))

heading: L = αLGtr + (1− α)LGrqtr , (4)
text: where pG(eq, rq, ea) represents the score of a positive triple within the specific KG G. The set {(eq, rq, ei)}ni=1 comprises negative samples, which are generated by corrupting either the head or the tail entity in the positive triple. The hyperparameter α balances two parts of loss. In practical scenarios, we manage two pre-training tasks through controlled batch sampling. With a probability of (1-α), we sample batches with identical query relations for low-resource pretraining, and otherwise, we sample regular batches for pretraining on the original KG.

heading: 3.2 LLM-based Prompter
text: To improve the GNN reasoner’s accuracy in lowresource scenarios, we create a prompt graph Gp
connecting few-shot relations with others according to semantic features, thereby filling in the gaps of the topological relation graph Gr . This is achieved by a frozen Large Language Model (LLM) extracting relation semantics from concise textual information. This graph prompting process requires no model fine-tuning, preserving the generalizability across distinct KGs.
Initial trials showed that asking the LLM for all possible relational interactions was inefficient and led to inaccuracies, involving too many requests and often resulting in flawed outputs. To overcome this, we streamlined the process by using the LLM to determine potential entity types for the heads and tails of relations. When two relations have matching entity types at their head sides, we assume an h2h interaction between them (similarly for other interaction types). This strategy cuts down LLM queries to a single one for each relation, simplifying the task and enhancing accuracy.
Instruction Prompt Design: To obtain entity types of two sides per relation, we design a series of instruction prompts, ensuring detailed guidance for each query relation. The prompt template is defined as P(·), and I = P(DRs , Let) is the input message to the LLM. Rs is the set of query relations with relation information DRs . The list of candidate entity types Let is utilized to control the range of LLM responses. As shown in Figure 4, these prompts are distinct in two aspects: (1) Relation Information Form:
• des: Short textual description of one relation. • exp: Textual entity names of one support triple. • d&e: Both description and example.
(2) Output Entity Type: • fixed: Limited to predefined entity types. • refer: Allow new types besides predefined. • free: No constraints on the type range.
As prompt examples in Table 8 in the Appendix, the semantic information required for each relation is concise, facilitating user editing or automatic generation. The list of candidate entity types is domain-dependent; for a general KG, it includes types like person, location, and event. When the type category is free, LLM would output any reasonable entity types with no constraints.
Prompt Graph Construct: Collecting responses from the LLM, we obtain the set of entity types S(r, ‘h’) and S(r, ‘t’) for the head and tail sides of each relation r. Then, we construct the prompt graph Gp = {R,Rfund , Tp}, where the re-
lation interactions in Tp obeys the following rules:
S(r1, s1) ∩ S(r2, s2) ̸= ∅ =⇒ Tp [r1, r2, s1, s2] = 1; Tp [r1, r2, s1, s2] = Tp [r1, r′2, s1, s′2] = Tp [r′1, r2, s′1, s2];
Tp [r1, r2, s1, s2] = { Tp [r2, r1, s1, s2] s1 = s2Tp [r2, r1, s′1, s′2] s1 ̸= s2 ;
where r1, r2 ∈ R and the side symbol si ∈ {‘h’, ‘t’}. The inverse relation of ri is denoted as r′i, and s ′ i denotes the opposite side of si. The connection value Tp [r1, r2, ‘h’, ‘t’] is equal to one when there is an h2t edge between r1 and r2, otherwise zero. The first rule specifies a sufficient condition for establishing relation interactions via entity types, while the other two rules detail the equivalence of inverse relations and inverse interactions. These rules collectively serve to minimize redundant computations in constructing Gp .

heading: 3.3 Prompt Calibrator
text: This component aims to improve the quality of the LLM-based prompt graph Gp leveraging the ground-true information in the topological relation graph Gr . Due to the natural gap between relation semantics and graph-specific topology, Gp cannot cover all expected interactions inevitably. Besides, mistaken edges would be included due to the relatively loose construction rules and the uncertainty of LLM response quality. Therefore, we design a novel calibrating process to extract high-confidence prompting edges that link the query relation with other relations in the KG. As shown in Algorithm 1, for each few-shot relation type rq, two learningfree mechanisms are utilized to extract a series of calibrated interaction edges. When inferring rqinvolved queries, these interaction edges would be injected into Gr to form the final rq-specific prompt graph Ĝrqr . Detailed descriptions and several examples are introduced in Appendix B.
Few-shot Support Expanding: In few-shot scenarios, support triples of this query relation in Gr are valuable. Therefore, we leverage these triples
Algorithm 1: Prompt Graph Calibrating Input :relation graph Gr , prompt graph
Gp , query relation rq, threshold β. Output :calibrated prompt graph Ĝr .
1 Gather rq-related edges from two graphs: T rqr ← {(ei, r, ej) ∈ Tr | ei = rq ∨ ej = rq} T rqp ← {(ei, r, ej) ∈ Tp | ei = rq ∨ ej = rq}
2 Expand potential edges: T rqex ← SupportExpanding(rq, Tr ) 3 Identify unmatched edges in Gp : Tm ← (Tp − T rqp )− (Tr − T rqr )
4 Filter prompting edges of rq: T rqconf ← ConflictFiltering(T rq ex ∪ T rqp , Tm , β) 5 Inject prompting edges into Gr : Ĝrqr ← {R,Rfund , T rqconf ∪ Tr}
to find more potential interaction edges via the following rules:
∃rj ,(rq, ir, rj) ∈ Gr , ir = [s1, s2] =⇒ S(rq, s1)← S(rq, s1) ∪ S(rj , s2)
∃rk,(rq, ir1, rj), (rj , ir2, rk) ∈ Gr , ir1 = [s1, s2], ir2 = [s2, s2] =⇒ Tp [rq, rk, s1, s2] = 1;
where ir = [s1, s2] denotes an interaction edge whose type is ‘s1-to-s2’ (e.g., ‘h2t’). Given the relations involved by support triples, the first rule inserts their entity types into the type set of rq, thereby expanding more potential edges. The second rule adds new interaction edges directly by finding the 2-hop neighbors of the query relation in Gr . These new edges T rqex as well as original rq related edges T rqp will be filtered in the next step.
Type Conflict Filtering: In a specific KG, two relations sharing an entity type in semantics may link to two disjoint entity sets, thereby the interaction edge between them is mistaken. Because rq lacks support triples, we can only detect whether the relations Rs, having the same entity type as rq at side s, have conflicts with each other. Therefore, we compare the Rs-involved edges in Gr and Gp , and extract unmatched edges Tm. If a relation has many unmatched edges with other relations in Rs, it is less likely connected with rq. The conflict determination of each relation is defined as follows:
C(rj) = |{rj |(rj , i, rk) ∈ Tm, rj , rk ∈ Ep}| ≥ β
The threshold hyperparameter β determines the maximum accepted number of unmatched edges for each relation rj . If C(rj) is true, the interaction edges connecting rj and rq would be removed.

heading: 4 Experiments
text: We extensively evaluate our method on K-shot inductive reasoning of KGs. In particular, we wish to answer the following research questions: RQ1: How effective is our method under the low-resource scenarios of distinct KGs? RQ2: How do different LLM prompts impact prompt graphs and KG inductive reasoning? RQ3: How do the main components of our method impact the performance? RQ4: How does our method perform in more shots and full-shot scenarios? RQ5: How efficient is our model compared with traditional approaches? RQ6: How does GNN reasoning change before and after injecting the prompt graph in case studies? Due to the space limitation, discussions about RQ5 and RQ6 are detailed in the Appendix.

heading: 4.1 Experimental Setup
text: Low-resource Datasets. We conduct K-shot inductive reasoning experiments on 108 low-resource datasets, constructed upon the 12 datasets used by the InGram work (Lee et al., 2023). The InGram datasets were derived from three real-world KG benchmarks: FB15k237 (Toutanova and Chen, 2015), Wikidata68K (Gesese et al., 2022), and NELL-995 (Xiong et al., 2017), abbreviated as FB, WK, and NL. There are four KG datasets (we call v1-v4) for each benchmark. For FB and NL datasets, we utilize the word-segmented relation names as the short description, while extracting official relation descriptions from WikiData for WK datasets. Both structural and textual statistics for these datasets can be found in Table 6 and Table 7 in the Appendix. Based on the InGram datasets, we create K-shot datasets tailored for 3-shot, 1-shot, and 0-shot scenarios. In each InGram dataset, we randomly retain K support triples for each query relation rq and mask others when reasoning from Grqinf. Ensuring a robust evaluation of our model, we perform this sampling process three times to create three variants for each few-shot dataset.
Baselines and Implementation. We compare PROLINK with eight baseline methods. NBFNet (Zhu et al., 2021), RED-GNN (Zhang and Yao, 2022), InGram (Lee et al., 2023), DEqInGram (Gao et al., 2023), and ISDEA (Gao et al., 2023) are state-of-the-art GNN-based inductive models. The recent pre-trained model ULTRA(Xg) (Galkin et al., 2023) has three variants (3g, 4g, 50g), of which X denotes the amount of pre-trained KG datasets. Previous few-shot link prediction and text-based methods are neglected because they can-
not perform on our task settings. For PROLINK, we train the GNN reasoner following the settings of ULTRA(3g) and employ five LLMs in the LLM prompter, including Llama2-7B, Llama2-13B (Touvron et al., 2023), Mistral-7B (Das et al., 2017), GPT-3.5 (Brown et al., 2020), and GPT-4 (OpenAI, 2023). We utilize two evaluation metrics, MRR (Mean Reciprocal Rank) and Hits@N. Hyperparameters are selected via grid search according to the metrics on the validation set. All experiments are performed on Intel Xeon Gold 6238R CPU @ 2.20GHz and 4× NVIDIA RTX A30 GPUs. Implementation details and hyperparameter configurations are shown in Appendix A.

heading: 4.2 Main Experimental Results (RQ1)
text: We compare PROLINK with baselines on inductive reasoning tasks under the 3-shot, 1-shot, and 0-shot settings. We report the average Hits@10 results over three variants of each K-shot InGram dataset in Table 1, Table 2, and Table 3. More detailed results on other metrics can be found in the Appendix. We observe that the first four GNN-based methods underperform pre-trained ULTRA and our method. NBFNet and REDGNN struggle with unseen relations, while InGram, ISDEA, and DEqInGram rely on sufficient support triples to form a similar distribution of node degrees. Conversely, ULTRA excels in low-resource settings due to its pre-training on multiple KGs, especially in the 3- shot scenario, though more KG pre-training in ULTRA(50g) doesn’t markedly boost performance.
Pre-trained on three KGs like ULTRA(3g), PROLINK significantly outperforms previous methods by a wide margin on average. Notably, in the 0- shot setting, the average Hits@10 for Our(GPT4) method is twice as high as that of ULTRA, highlighting the effectiveness of our prompting paradigm. In the comparison of different LLMs, GPT-3.5 and GPT-4 exhibit superior performance in the 0-shot setting. Due to our simplification for LLM requests, the lightweight Llama2-7B already performs well in few-shot scenarios, which indicates the robustness of our method.

heading: 4.3 Performance of Different Prompts (RQ2)
text: To generate the prompt graph, we employ a series of textual prompts to guide the LLM prompter. As shown in Figure 5, we compare the performance of Our(GPT-4) on the three v4 datasets by varying the Relation Information Form and Output Entity Type, respectively. The ‘d&e’ form outperforms the other two in most scenarios, because textual descriptions reflect the core features while two entity names indicate the direction of the relation. Regarding entity types, the ‘free’ setting works better in fewshot scenarios. In zero-shot datasets, prompts using (‘d&e’, ‘fixed’) outperform the others.
In Figure 6 in Appendix, we analyse the effectiveness of the prompt graph by calculating the F1 metric between the prompt interaction edges and ground-truth edges in the raw KGs. The trend of F1 metrics across different prompts is similar to that of Hits@10, it shows that a prompt graph closer to
ground truth leads to better performance. The results indicate the importance of prompt settings and more refined prompts would be our future work.

heading: 4.4 Ablation Studies (RQ3, RQ4)
text: To validate the impact of the three components on model performance, we conduct ablation experiments on PROLINK with GPT-4, as shown in Table 4. The variant ‘with ULTRA’ employs ULTRA(3g) to replace our pre-trained GNN reasoner and the performance decline indicates the effect of our enhancing techniques. ‘w/o Prompt’ denotes utilizing the pre-trained KG reasoner directly without prompt graphs, which performs well in 3-shot scenarios but struggles with fewer-shot ones. The next two variants prove that the LLM prompter and the prompt calibrator both enhance the model’s performance. ‘with OODKG’ denotes pre-training the GNN reasoner with three KGs out of the evaluation KG distribution, including YAGO3-10 (Mahdisoltani et al., 2015), DBPedia100k (Ding et al., 2018), and WN18RR (Bordes et al., 2014) Its competitive performance indicates the robustness of our pre-training strategy.
Regarding RQ4, PROLINK consistently surpasses ULTRA(3g) in most cases of 2/4/8/16 shots experiments. As the full results presented in Table 9 in Appendix, the average performance improvement is still significant in more-shot scenarios, which validates our model’s superiority. We also verify PROLINK in the full-shot scenarios. Our(GPT-4) and Our(Llama2-7b) outperform UL-
TRA(3g) and InGram on FB:v4 and NL:v4 datasets. The results prove the good applicability of our method in practice.

heading: 5 Conclusions
text: We propose a novel pre-training and prompting framework, PROLINK, for low-resource inductive reasoning across arbitrary KGs. To generate an effective prompt graph for few-shot relation types, we design enhancing techniques for the GNN reasoner and instruction prompts for the LLM prompter. Besides, a novel prompt calibrator is proposed to mitigate the potential noise and achieve the information alignment of the above two components. Extensive experiments have verified that the PROLINK achieves significant performance in both few-shot and zero-shot scenarios. Besides, PROLINK requires no model fine-tuning, thereby having advantages of better efficiency and scalability than previous GNN-based methods.

heading: 6 Limitations.
text: Here, we discuss two potential limitations of PROLINK. First, unlike recent GNN-based fully inductive methods, our method requires a brief relational context for each relation type. Despite our efforts to minimize text requirements, it may be unavailable in certain scenarios without manual inputs from users. Second, the usage of textual context in our method is not sufficient enough. To circumvent the need for additional model training, we streamline the LLM queries by only asking for entity types. Incorporating semantic embeddings into GNNs or fine-tuning the LLM prompter to leverage relation semantics could further enhance performance. Exploring this avenue will constitute a primary focus of our future work. The potential risks of our work may include generating factual triples about privacy or fake information, depending on the legality and reliability of the input data. In addition, we utilize the ChatGPT AI assistant when polishing some paragraphs of the paper draft.

heading: Acknowledgments
text: This research is supported by Singapore MOE funding (RS05/21, MOE-T2EP20122-0003) and NTUNAP startup grant (022029-00001).

heading: B Detailed Discussion about Prompt Calibrator
text: Here, we further introduce the algorithm in prompt calibrator, accompanied by specific examples. This algorithm aims to extract high-confidence prompting edges while minimizing potential noise edges by harnessing ground-truth information from the existing topological relation graph.
The mechanism for few-shot support expanding focuses on the ground-truth interaction edges of the target few-shot relation rq. This process follows two rules, enabling the inference of more potential edges from these ground-truth edges. The first rule extends the entity type set of rq by incorporating the entity types of its linked relations. For instance, in Figure 1, if we observe that the ‘co-worked’ relation has a ‘h2t’ edge to ‘sung by’, the entity types on the tail side of ‘sung by’ become more likely to be the head entity type of ‘co-worked’. These extended entity types facilitate the deduction of more potential edges according to the rules in prompt graph construction. The second rule employs a specific 2-hop pattern. For instance, because ‘sung by’ and ‘lyricized by’ are linked by a ‘t2t’ edge, they are likely to share at least one same entity type on the tail side. Consequently, we can infer that ‘co-worked’ may have a ‘h2t’ edge to ‘lyricized by’. However, these expanding edges remain uncertain and necessitate further noise filtering.
The type conflict filtering mechanism centers on the ground-truth interaction edges of other relations except rq. As all entity types for each relation are predicted by the LLM prompter, prompting edges among these full-shot relations are also accessible. Since full-shot relations are presumed to encompass all interaction edges, disparities between prompting edges and ground-truth edges can indicate the accuracy of edge type prediction. Hence, given a set of relations sharing the same entity type as rq, we can analyze the mistaken prompting edges among them and assess whether the entity range of each relation is disjoint from that of others. Such relations with conflicts are more likely to have no
4https://github.com/DeepGraphLearning/ULTRA
connections with rq. For example, ‘directed by’ and ‘lyricized by’ share the same tail entity type ’people’, but they do not share the same tail entities. Consequently, these two relations are hardly to be simultaneously linked to ‘sung by’.

heading: C Efficiency Analysis (RQ5)
text: The primary benefit of PROLINK lies in its ability to infer new, arbitrary Knowledge Graphs (KGs) without the need for training. It stems from the fact that both the GNN reasoner and the LLM prompter remain parameter-frozen during the prompting process. As a result, the only computational overhead introduced is associated with the prompt graphs. Except for the LLM requests for each relation in the KG, the prompt graph is generated only once for each few-shot relation type, whose computational complexity would not exceed O(|Rinf|2). Given that the quantity of relations in most KGs is significantly lower than that of entities, the time and space costs in this process are negligible. Specifically, the checkpoint file size of our KG reasoner is only 2.18 MB, which is similar to that of ULTRA (2.03 MB). The total pretraining time is around eight GPU hours for ten training epochs. The evaluation time on each dataset only costs several minutes. Regarding the model scalability, PROLINK can be applied in large-scale KGs the same as ULTRA
(Galkin et al., 2023). This work focuses on lowresource challenges, more experiments on largescale datasets will be our future work.

heading: D Case Studies (RQ6)
text: We select several queries from the zero-shot NL:v4 dataset and compare the outputs of Our(GPT-4) with and without the prompt graph. The final relation graphs and top five outputted entities are shown in Figure 7. Generally, we observe that PROLINK injects multiple prompt edges into the relation graph which directly changes the outputted entity ranking. Most linked relations by prompt edges are reasonable, thereby improving the relative relation embedding for the query relation.

heading: E Detailed Related Work
text: KG Inductive Reasoning: Traditional transductive KG embedding models, represented by TransE (Bordes et al., 2013), DistMult (Yang et al., 2015) and RotatE (Sun et al., 2019), learn continuous vectors in the embedding space to represent each entity and relation in the knowledge graph (Wang et al., 2017, 2020, 2022). However, these models lack
generalization capabilities for new entities within the same graph or across different KGs (Wang et al., 2021c; Chami et al., 2020). In contrast, inductive KG reasoning methods (Zhu et al., 2021) overcome this limitation by generalizing to KGs with unseen entities or relations. Most existing inductive methods (Yan et al., 2022; Wang et al., 2021a; Liu et al., 2023a; Chen et al., 2021) leverage Graph Neural
Networks (GNNs) to generate “relative” entity embeddings, by extracting local structural features from an induced graph of the query entity. GraIL (Teru et al., 2020) extracts an enclosing subgraph between the query entity and each candidate entity, but suffers from high computational complexity. NBFNet (Zhu et al., 2021) and RED-GNN (Zhang and Yao, 2022) propagate query features through the L-hop neighborhood subgraph of the query entity. To further enhance model efficiency, recent studies have focused on algorithmic improvements, including path-pruning methods in the GNN propagation process (Zhang et al., 2022; Zhu et al., 2023b; Wang et al., 2023). These inductive methods struggle to generalize to KGs with new relation types, as the entity embeddings are still a function of a predetermined relation vocabulary.
Few-shot and Unseen Relation Reasoning: To generalize to unseen relations, early efforts have explored meta-learning for few-shot link prediction, which predicts KG facts of unseen relations using a limited number of support triples (Chen et al., 2019; Zhang et al., 2020; Huang et al., 2022; Pei et al., 2023; Wu et al., 2023). However, those methods cannot work on an entire unseen inference graph. Recent approaches focus on constructing graphs of relations to generalize to unseen relations (Geng et al., 2023a; Zhou et al., 2023). RMPI (Geng et al., 2023a) transforms the surrounding subgraph of a triple in the original KG into a new relation-view graph, then learns the embedding of an unseen relation from the relational subgraph. InGram (Lee et al., 2023) relies on a featurization strategy based on the discretization of node degrees, which is effective only for KGs with a similar relational distribution and falls short in transferring to arbitrary KGs. ISDEA (Gao et al., 2023) utilizes relation exchangeability in multi-relational graphs, which imposes stringent assumptions on the class of transferable relations coming from the same distribution. ULTRA (Galkin et al., 2023) employs a pretraining and fine-tuning framework, with a global relation graph extracted from the entire set of KG
triples. However, the effectiveness of the above methods is contingent on sufficient support triples for the unseen relations in the inference KG, struggling with few-shot relation types in low-resource scenarios that limit their applicability.
Text-based methods via Language Models: In recent years, the AI community has witnessed the emergence of numerous powerful pre-trained language Models (PLMs), such as BERT (Devlin et al., 2019), Llama-2 (Touvron et al., 2023), and GPT4 (OpenAI, 2023) which are driving huge advancements and lead to the pursuit of possible Artificial General Intelligence (AGI) (Ouyang et al., 2022; Bubeck et al., 2023). A line of text-based KG inductive reasoning methods like BLP (Daza et al., 2021), KEPLER (Wang et al., 2021e), StATIK (Markowitz et al., 2022), RAILD (Gesese et al., 2022) rely on textual descriptions of entities and relations and use pre-trained language models (PLMs) like BERT (Devlin et al., 2019) to encode them. To seek higher prediction performance, current meth-
ods mostly prefer to fine-tune PLMs which requires high computational complexity and limits its generalizability to other KGs with different formats of textual descriptions. CSProm-KG (Chen et al., 2023) and PDKGC (Geng et al., 2023b) perform KG link prediction via a frozen PLM with only the prompts trained. By combining graph structure features and textual representations, these methods perform well in transductive KG reasoning tasks. However, the trainable prompts are tuned for a fixed relation vocabulary and cannot be generalized to inductive reasoning. Recent work (Zhu et al., 2023a) verifies the reasoning performance of large language models by converting a query into textual names of the query entity and relation. We consider this family of methods to be orthogonal to our work. Our approach operates under the lowresource assumption that the graphs lack explicit entity descriptions, focusing instead on leveraging structural graph information and relation descriptions.

heading: Future Work
text: This work focuses on lowresource challenges, more experiments on largescale datasets will be our future work