## 2 Related Work
### Pre-Trained Language Model- and Large Language Model-Based Methods  
Based on the self-attention mechanism, pre-trained language models are constructed and are pre-trained on a large corpus of text. These models have learnt vast amounts of knowledge and can be well adapted to a variety of downstream tasks through simply fine-tuning, such as machine translation [10], text analysis [11], and intelligent question answering [12]. Some studies also utilize the pre-trained language models to enhance the performance of knowledge graph reasoning. KG-BERT [28] represents the query triple as a textual sequence with the special token [CLS] in front of the sequence. Then, the sequence is encoded by the pre-trained language model BERT [29] and the hidden state of [CLS] is put into a classifier to predict the missing entity. KGT5 [30] further trains T5 small [31] on link prediction and question answering tasks with multiple forms; then, the reasoning can be completed by integrating the query triple and simple instructions. GenKGC [32] is developed based on the model BART [33]. Besides the query triple, the method also incorporates demonstration triples which contain the same query relation to improve the reasoning results.  
Subsequently, large language models are proposed and bring new waves in multiple fields. LLMs are trained on a very large corpus and contain hundreds of billions of parameters. Many experiments have proven that LLMs perform well at various natural language processing tasks without further training for the tasks [10; 11; 12]. In order to make the LLMs comprehend different downstream tasks, in-context learning is widely used [13]. It is a type of few-shot learning method which incorporates a few examples into the prompt [15]. AutoKG [34] utilizes the LLMs GPT4 and chatGPT ([[https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt)]([https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt)) (accessed on 15 January 2025)), and adopts in-context learning on knowledge graph reasoning through one-shot learning. A triple is extracted as the context example to enhance the reasoning performance. KAPING [35] first links the entities in the query and that of knowledge graphs, and the triples which contain the linked entities are retrieved as in-context triples. KoPA [36] selects the triples which contain the query head entity as in-context learning triples to enhance the performance of triple classification. KICGPT [37] first conducts the knowledge graph reasoning through the distributed representation learning method RotatE, and the results are further refined by the LLMs through multiple interactions.  
However, the in-context triples utilized in previous LLM-based reasoning methods are mainly retrieved around the query entities or query relations, which cannot make full use of the existing triples of knowledge graphs. The triples that are not directly connected to the query entities, as well as those that do not contain the query relations, may also potentially provide important knowledge for reasoning. While our proposed method can retrieve these useful in-context triples with the guidance of logical rules and the feedback of LLMs, the experimental results also demonstrate the effectiveness of the model.