Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs

Authors: liyi  chen
panrong  tong
zhongming  jin
ying  sun
jieping  ye
hui  xiong
taylor  swift
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-11 03:59:40

Abstract: Large Language Models (LLMs) have shown remarkable reasoning capabilities on complex tasks, but they still suffer from out-of-date knowledge, hallucinations, and opaque decision-making. In contrast, Knowledge Graphs (KGs) can provide explicit and editable knowledge for LLMs to alleviate these issues. Existing paradigm of KG-augmented LLM manually predefines the breadth of exploration space and requires flawless navigation in KGs. However, this paradigm cannot adaptively explore reasoning paths in KGs based on the question semantics and self-correct erroneous reasoning paths, resulting in a bottleneck in efficiency and effect. To address these limitations, we propose a novel self-correcting adaptive planning paradigm for KG-augmented LLM named Plan-on-Graph (PoG), which first decomposes the question into several sub-objectives and then repeats the process of adaptively exploring reasoning paths, updating memory, and reflecting on the need to self-correct erroneous reasoning paths until arriving at the answer. Specifically, three important mechanisms of Guidance, Memory, and Reflection are designed to work together, to guarantee the adaptive breadth of self-correcting planning for graph reasoning. Finally, extensive experiments on three real-world datasets demonstrate the effectiveness and efficiency of PoG.

Main paper: 
 introduction: 
 Large Language Models (LLMs) have manifested outstanding performance in various natural language processing and data science tasks, such as question answering [42, 25, 62] , text generation [18, 13, 8, 15] , recommender systems [60, 59, 51, 44] , and domain-specific applications [46, 45, 14, 50, 36] . They leverage advanced deep learning techniques and immense amounts of pre-existing text data to understand and generate human language with impressive fluency and coherence. Despite their success in numerous applications, LLMs still suffer from out-of-date knowledge, hallucinations, and opaque decision-making, highlighting the ongoing need for further investigation in this rapidly evolving field.Intuitively, as large-scale structural knowledge bases, Knowledge Graphs (KGs) [5, 1, 12, 43] provide explicit and editable depictions of massive real-world knowledge, which have the potential to be a promising complement to the drawbacks of LLMs. Previous studies manage to integrate KGs into LLM pre-training [61, 47] or fine-tuning [53, 27] stage. However, these methods mainly compress structured knowledge in KGs into LLMs' parameters in a black-box fashion and still cannot fully enhance the flexibility, reliability, and transparency of LLMs. Therefore, several attempts [3, 2, 40] first retrieve information from KGs and then deliver explicit knowledge into LLMs. Under these circumstances, LLMs do not directly participate in the graph reasoning process, making these methods excessively dependent on the KG completeness. Recently, a KG-augmented LLM paradigm has been proposed to conduct graph reasoning, which treats the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. For instance, StructGPT [19] and ToG [35] predefine the breadth of reasoning paths explored on the KG, and leverage the LLM to iterate the process of unidirectionally extending along the reasoning paths relevant to the question and reasoning the answer using these reasoning paths. This KG-augmented LLM paradigm offers an opportunity for more comprehensively amalgamating the knowledge from both the KG and LLM by facilitating the step-by-step derivation of further insights.However, existing paradigm may fail to plan the exploration of correct reasoning paths for many complex questions. Figure 1 illustrates an example of existing paradigm's limitations when answering the question "Which of Taylor Swift's songs has won American Music Awards? (AMA)". These limitations lie in: (1) Predefined path breadth: Existing paradigm requires manually setting the breadth of reasoning paths in KGs, and a fixed breadth may result in all the selected relations or entities being incorrect. When determining the relevance between paths and questions in step 2 ⃝, due to the limited maximum breadth of three and the uncertainty surrounding the respective awards of songs, the LLM selected maximum numbers of songs, ignoring the correct entity "Blank Space". (2) Irreversible exploration direction: The path exploration in existing paradigm is unidirectional without the ability to self-correct. Even if the paths are incorrect, the LLM still continues to extend current incorrect paths and lead to the failure of reasoning on the KG. In step 3⃝ and 4 ⃝, since "The Joker And The Queen", "Shake It Off", and "Cruel Summer" were already chosen, the reasoning process continued on incorrect paths and the right answer was not found. (3) Forgetting partial conditions: During reasoning, the LLM may forget partial conditions in the question and cannot provide the answer that satisfies multiple conditions simultaneously. In step 4⃝, the LLM only remembered the condition that the song was by Taylor Swift but forgot the condition about the song winning an AMA award, leading to an incorrect answer, "Love Story". Therefore, the reasoning of complex questions may heavily rely on adaptive exploration and self-correction of erroneous reasoning paths.To address these limitations, we propose a novel self-correcting adaptive planning paradigm for KGaugmented LLM named Plan-on-Graph (PoG). To the best of our knowledge, we are the first to design a reflection mechanism for self-correction and adaptive KG exploration into KG-augmented LLMs, effectively improving the ability and efficiency of LLM reasoning. Specifically, PoG first decomposes the question into several sub-objectives as guidance for planning exploration, and then repeats the process of adaptively exploring reasoning paths to access relevant KG data, updating memory to provide dynamic evidence for reflection, and reflecting on the need to self-correct reasoning paths until arriving at the answer. In PoG, three mechanisms are designed for adaptive self-correcting planning: (1) Guidance: To better guide adaptive exploration by harnessing conditions in the question, we employ the LLM to decompose the question into sub-objectives containing conditions, thereby benefiting the identification of relevant paths to each condition with flexible exploration breadth. (2) Memory: The information stored in memory offers historical retrieval and reasoning information for reflection. We record and update the subgraph to provide the LLM with all retrieved entities for initializing new exploration and self-correcting paths, reasoning paths to preserve the relationships between entities for LLM reasoning and allow for path correction, and sub-objective status to make the LLM recognize the known information of each condition and mitigate its forgetting in reflection stage. (3) Reflection: To determine whether to continue or self-correct current reasoning paths, we design a reflection mechanism to employ the LLM to reason whether to consider other entities into new exploration and decide which entities to backtrack to for self-correction based on information in memory. Finally, extensive experiments on three real-world KGQA datasets validate the effectiveness and efficiency of PoG1 . The main contributions of this paper are listed as follows:• We propose a novel self-correcting adaptive planning paradigm for KG-augmented LLM named PoG, which exploits the LLM to plan the adaptive breadth of reasoning paths and reflect to self-correct erroneous paths. To the best of our knowledge, we are the first to incorporate a reflection mechanism for self-correction and adaptive KG exploration into KG-augmented LLMs, effectively augmenting the LLM's reasoning ability.• We specially design Guidance, Memory, and Reflection mechanisms for PoG. Guidance harnesses question conditions to better plan adaptive exploration by decomposing task into sub-objectives including conditions. Memory records the subgraph, reasoning paths, and sub-objective status to provide historical retrieval and reasoning information for Reflection.Based on Memory, Reflection reasons whether to self-correct reasoning paths and which entity to backtrack to for initiating new exploration.• We conduct extensive experiments on three real-world KGQA datasets, namely CWQ, We-bQSP, and GrailQA. The results demonstrate not only the effectiveness but also the efficiency of our proposed novel PoG paradigm for KG-augmented LLM.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: Please see the abstract and introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
preliminary: 
 Knowledge Graph (KG) stores massive factual knowledge in the form of a set of triplets: G = {(e, r, e ′ ) | e, e ′ ∈ E, r ∈ R}, where E and R denote the set of entities and relations, respectively.Relation Paths are a sequence of relations: z = {r 1 , r 2 , ..., r l }, where r i ∈ R denotes the i-th relation in the path and l denotes the length of the path.Reasoning Paths are the instances of a relation path z in the KG:p z = e 0 → r 1 e 1 → r 2 e 2 → ... → r l e l, where e i ∈ E denotes the i-th entity and r i denotes the i-th relation in the relation path z.Knowledge Graph Question Answering (KGQA) is the task of answering natural language questions based on a set of facts over the KG. Given a question q, a knowledge graph G, and topic entities T q mentioned in q, the target of KGQA is to generate answers A q to the question q. Following previous studies [35] , we assume any entity e q ∈ T q mentioned in q and answers a q ∈ A q are labeled and linked to the corresponding entities in G, i.e., T q , A q ⊆ E.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations in the Appendix H.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methodology: 
 In this section, we introduce the technical details of the novel self-correcting adaptive planning paradigm for KG-augmented LLM named Plan-on-Graph (PoG). As illustrated in Figure 2 , PoG consists of four key components: Task Decomposition, Path Exploration, Memory Updating, and Evaluation. PoG first decomposes the question into several sub-objectives as guidance of planning exploration and then repeats the process of adaptively exploring reasoning paths to access relevant KG data, updating memory to provide historical retrieval and reasoning information for reflection, and reflecting on the need to self-correct reasoning paths until arriving at the answer. "The Joker And The Queen", "Shake It Off", and "Cruel Summer" have not won AMA, so I need to find other songs. I add the songs "Blank Space", "Love Story", … into the following exploration. To harness conditions in the question to better guide the adaptive exploration process, PoG decomposes the task of answering the question into multiple sub-objectives containing conditions through semantic analysis of the LLM. Sub-objectives serve as guidance for path exploration, benefiting the identification of relevant paths to each condition outlined in the question with flexible exploration breadth. Specifically, we prompt the LLM to decompose the original question q into a list of sub-objectives for KG retrieval and reasoning. The prompt is shown in Appendix A.1. The list of sub-objectives can be denoted as O = {o 1 , o 2 , o 3 , ...}. It is important to note that sub-objectives in O may refer to the results obtained from other sub-objectives in O, allowing for interdependencies in the reasoning process.We access relevant information from the KG by exploring reasoning paths in the KG. On the initiation of path exploration, we localize the initial entities of reasoning paths, which correspond to the topic entities mentioned in the given question. Similar to prior research [19, 35] , topic entities have been pre-identified and are part of the annotated datasets. Specifically, when presented with a question q, we use topic entities to serve as the initial elements of the reasoning paths, E 0 = T q = {e 0 1 , e 0 2 , ..., e 0 N0 }, where N 0 is the number of topic entities.In the subsequent iterations, we continue exploring reasoning paths most relevant to the question and suspend other reasoning paths. Taking the D-th iteration as an example, before the iteration starts, each reasoning path p n ∈ P consists of D pn (D pn ≤ D -1) triplets, i.e., p n = {(e d s,n , r ) are linked to each other. It is noted that the length of each reasoning path may vary, because in the D-th iteration, we only continue exploring the reasoning paths most semantically relevant to the question, which are identified in the D -1-th iteration. The sets of tail entities and relations to be explored are denoted asE D-1 = {e D-1 1 , e D-1 2 , . . . , e D-1 N D-1 } and R D-1 = {r D-1 1 , r D-1 2 , . . . , r D-1 N D-1 }, respectively, where N D-1 is the length of E D-1 and R D-1 . We leverage the LLM to identify the most relevant entities E D from the neighboring entities of the current entity set E D-1 based on the question q and extend the reasoning paths P with E D . In order to manage the complexity of dealing with a large number of neighboring entities using the LLM, we propose an adaptive exploration strategy that is not limited by the fixed number of relations and entities. This strategy involves a two-step process of finding relevant relations and utilizing these selected relations to explore entities.Relation Exploration. Relation exploration is a process to retrieve the relations of all tail entities in E D-1 and identify the most relevant relations to the question q and the sub-objectives O. To be specific, we first conduct the search to obtain all relations linked to the tail entities in E D-1 as the candidate relation set R D cand = {r D cand,1 , r D cand,2 , ..., r D cand,N D-1 }. We utilize R D cand to extend the reasoning paths into candidate reasoning paths P cand . Then, we employ the LLM to select a flexible number of relevant reasoning paths P ending with the tail relations in R D from P cand , based on the semantic information of the question q, tail entities E D-1 , candidate relations R D cand , and sub-objectives O. The prompt is shown in Appendix A.2.1, and the pre-defined query for relation search is shown in Appendix B.1.Entity Exploration. Analogously, entity exploration is a process to retrieve neighboring entities based on R D and E D-1 and detect the most relevant entities to the question q. From the previous relation exploration, we obtain extended reasoning paths P and new tail relations R D . For each reasoning path p n ∈ P , we can execute the queries of (eD-1 n , r D n , ?) or (?, r D n , e D-1 n ) to retrieve a candidate entity set E D cand,n, where e D-1 n and r D n are the tail entity and relation in p n . When confronted with a large number of candidate entities, we use a small pre-trained DistilBERT [31] 2 , to calculate the similarity between candidate entities and the question for recall. Then, we summarize all candidate entity sets into E D cand and use E D cand as the tail entities to expand P into P cand . With the candidate reasoning paths P cand , we exploit the LLM to choose a flexible number of relevant reasoning paths P ending with the tail entities E D from P cand , based on the semantic information of the question q and knowledge triplets composed of tail entities E D-1 , tail relations R D and candidate entities E D cand . The prompt is shown in Appendix A.2.2, and the pre-defined query for entity search is shown in Appendix B.2.The information stored in memory provides historical retrieval and reasoning information for reflection. After a two-step exploration, we dynamically update the searched subgraph G Sub , reasoning paths P , and sub-objective status S in memory based on the ongoing reasoning process.Subgraph. The subgraph includes all retrieved relations and entities from the KG. We update the subgraph in memory, which can be utilized during later reflection to determine which entity to backtrack to for self-correction. In the D-th iteration, the searched subgraph G Sub is updated by adding the retrieved candidate relation set R D cand and candidate entity set E D cand . Reasoning Paths. In order to ensure that the LLM can understand relationships between entities for better reasoning and allow for path correction in reflection stage, we update reasoning paths P to preserve the semantic structure within the KG.Sub-Objective Status. The LLM may forget partial conditions in the reasoning process. Subobjectives obtained by decomposing the question can help the LLM remember multiple conditions in the question. The status of sub-objectives contains the current known information related to the sub-objectives, which can aid the LLM in remembering the known information of each condition and determining whether to correct the exploration direction in reflection stage. Hence, we leverage the LLM to update the currently known information relevant to sub-objectives into sub-objective status S = {s 1 , s 2 , s 3 , ...}, |S| = |O|, based on the semantic information of the question q, sub-objectives O, historical sub-objective status, and reasoning paths P , along with the LLM's own knowledge. The prompt is shown in Appendix A.3.After the path exploration and memory updating, PoG prompts the LLM to reason whether the current acquired information, including sub-objective states and reasoning paths recorded in memory, is sufficient to infer an answer. The prompt is shown in Appendix A.4.1. If the LLM determines that the information is sufficient, it will integrate reasoning paths, sub-objective states, and its own knowledge to provide an answer. When information is considered insufficient, there may be two situations. One is that PoG will acquire sufficient information after further extension of current paths, and the other is that current paths are incorrect. Since the reasoning capability of the LLM does not always guarantee the correctness of path exploration, there is a need to self-correct erroneous reasoning paths. Therefore, we design a reflection mechanism to determine whether and how to self-correct reasoning paths. When the LLM believes that the information is insufficient, PoG enters the stage of reflection. Specifically, PoG utilizes the LLM to reflect on whether to correct the current exploration direction based on the question q, sub-objective status S, reasoning paths P , and entities planned for the next iteration of retrieval E D from memory. Besides, the LLM will provide the reason for the reflection result. If the LLM judges that it is necessary to incorporate additional entities beyond those in E D for exploration, then a self-correction of reasoning paths is needed. Otherwise, PoG will continue exploring along the current reasoning paths with tail entities in E D . For self-correction, PoG employs the LLM to decide which entities inE cand = E 1 cand ∪ E 2 cand ∪ ... ∪ E Dcand to backtrack to based on sub-objective states in S and the reason for additional retrieval obtained from the reflection, and adds new exploration of backtracked entities E D add into E D for the self-correction, denoted asE D = E D ∪ E D add .The prompts for reflection are shown in Appendix A.4.2.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer • The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
experiments: 
 4.1 Experimental SetupsTo demonstrate the effectiveness of PoG on complex reasoning over knowledge graphs, we adopt three representative multi-hop KGQA datasets: CWQ [37] , WebQSP [56] , and GrailQA [17] . All three datasets rely on the external knowledge graph from Freebase [5] . For the large dataset GrailQA, we utilize the same testing samples as those in ToG [35] to improve computational efficiency. Following prior research [23, 19, 35] , we use exact match accuracy (Hits@1) as the evaluation metric.Due to variations in the performance of the method across different datasets, we select prior state-ofthe-art (SOTA) approaches as baselines for each dataset. They can be categorized into two groups:(1) LLM-only methods, including standard prompting (IO prompt) [6] , Chain-of-Thought prompting (CoT) [49] , and Self-Consistency (SC) [48] .(2) KG-augmented LLM methods, including fine-tuned and prompting methods. For CWQ and WebQSP, we utilize UniKGQA [20] , TIARA [34] , RE-KBQA [7] , DeCAF [57] ,and RoG [27] as fine-tuned baselines and KD-CoT [41] , KB-BINDER [23] , StructGPT [19] ,Interactive KBQA [52] , and ToG [35] as prompting baselines. For GrailQA, we utilize RnG-KBQA [55] , TIARA [34] , FC-KBQA [58] , Pangu [16] , FlexKBQA [26] , and GAIN [33] as fine-tuned baselines and KB-BINDER [23] and ToG [35] as prompting baselines. The descriptions of baselines are presented in Appendix D.We compare PoG with the SOTA baselines to demonstrate its effectiveness for KG-augmented LLM. Table 1 and Table 2 present the experimental results on CWQ, WebQSP, and GrailQA datasets. Overall, PoG achieves the best performance across all three datasets. Specifically, we can make the following observations. First, compared to all prompting KG-augmented LLM baselines, PoG shows superior performance advantages. Regardless of whether GPT-3.5 or GPT-4 is used as the underlying LLM, PoG substantially outperforms the SOTA baseline, ToG. ToG explores reasoning paths with a fixed exploration breadth and cannot detect or correct the errors, showing limitations in effect and efficiency. Meantime, we specially design self-correction and adaptive planning mechanisms, which can effectively improve both performance and efficiency. Second, although PoG is a training-free prompting method, its performance is highly competitive with fine-tuned KG-augmented LLM baselines. When using GPT-4, the performance of PoG exceeds all fine-tuned KG-augmented LLM baselines across the board. Even with GPT-3.5, the result of PoG on GrailQA surpasses all fine-tuned KG-augmented LLM methods. This suggests that our designed guidance, memory, and reflection mechanisms allow PoG's effect to surpass most of the fine-tuned methods. Third, the improvement of PoG is obvious when compared to LLM-only baselines, which do not leverage external KGs. Besides, all KG-augmented LLM methods consistently outperform LLM-only methods, indicating the value of incorporating KGs to enhance LLM performance. Moreover, PoG further improves the effectiveness of KG-augmented LLMs through its self-correctable adaptive planning. Additionally, it is worth noting that PoG with GPT-3.5 outperforms other methods on the zero-shot subset of the GrailQA dataset by a large margin, apparently outperforming all fine-tuned KG-augmented LLMs on this category. The self-correction mechanism in PoG allows it to dynamically correct errors during the reasoning process, which is crucial for zero-shot problems. In order to assess the effectiveness of each mechanism and adaptive exploration in PoG, we conduct the ablation study to remove them on three datasets, respectively. Specifically, w/o Guidance refers to the variant where entire task decomposition as guidance is removed. w/o Memory indicates the variant without the memory mechanism. w/o Reflection refers to the variant where, in the case of insufficient information, it only continues exploring along the original reasoning paths. w/o Adaptive Breadth means that the variant uses a fixed exploration space breadth instead of adapting it based on the situation. Table 3 shows the performance of all variants and the results suggest that each mechanism and adaptive breadth appears to contribute positively to the overall performance, with their removal leading to weaker results on complex question answering tasks across the evaluated datasets. These variations achieve a minimum reduction of 3.0%, 2.1%, and 3.5% on CWQ, WebQSP, and GrailQA, respectively. The performance of w/o Memory drops the most, followed by w/o Reflection, because without the memory there is no information to support PoG in navigating the exploration and achieving self-correction, and PoG cannot self-correct the wrong reasoning paths without the reflection mechanism. Moreover, after setting a fixed maximum breadth for exploration, the performance deteriorates. This indicates that a fixed breadth makes the method lack flexibility and less adaptable to different questions. However, because the mechanisms of memory and reflection ensure that PoG is able to self-correct, the performance of w/o Adaptive Breadth does not decrease a lot.We study the efficiency of PoG and the SOTA prompting KG-augmented LLM baseline, ToG. Most importantly, PoG achieves superior time efficiency compared to ToG. On CWQ and GrailQA, PoG presents a speedup of over 4 times. ToG predefines the breadth of exploration, leading to the exploration of many irrelevant paths. Additionally, ToG lacks a self-correction mechanism, and when there is insufficient information to answer a question, it can only extend the current reasoning paths, sacrificing a lot of efficiency on irrelevant explorations. By contrast, the efficiency advantages of PoG can be attributed to its adaptive exploration and self-correction of reasoning paths based on the semantics of the question. The adaptive breadth reduces unnecessary exploration, and effective correction avoids the extending of wrong current paths.Figure 3 shows a typical case from the testing results on CWQ dataset. We compare the results of PoG, ToG, and CoT in answering the question "Who is in control of the place where the movie 'The Naked and the Dead' takes place?". The underlying LLMs they used are all based on GPT-3.5. PoG initially identifies a flexible number of relations related to the topic entities. Specifically, for "The Naked and the Dead", PoG successfully discovers that the movie takes place in Panama, while for "President of Panama", the LLM thinks that only the relation "government.government_office_or_title.jurisdiction" is relevant. Upon retrieval, no information is found regarding the person in control of Panama. This triggers reflection as PoG realizes that it lacks sufficient information. With the memory, PoG refers to the sub-objective status and recognizes that it already knows the movie location (Panama) for sub-objective #1 but is unaware of the person in control of Panama for sub-objective #2. Based on the current reasoning paths, PoG makes a decision to execute self-correction and returns to exploring the relation not previously explored for "President of Panama". Due to the task decomposition, during the self-correction process, it becomes easier to identify the correct relation "government.government_office_or_title.office_holders" according to the sub-objectives. Through the guidance, memory, and reflection mechanisms, PoG successfully finds the correct answer, "Juan Carlos Varela". In contrast, ToG fails to identify the most relevant relation concerning "President of Panama" and continues exploring incorrect paths. This consumes a significant amount of time and ultimately leads to an erroneous answer due to the hallucination. CoT refuses to answer directly since the LLM realizes its lack of knowledge regarding the answer and requires additional information to be provided. Answer: [Refused to answer.] I'm sorry, but I need to clarify the question a bit further in order to provide an accurate answer. Could you please specify whether you are referring to the character or the entity in control of the place where the movie "The Naked and the Dead" takes place? Thank you for your understanding.Answer: [KG exploration failed, and the LLM answered the question according to its own knowledge.] First, the movie "The Naked and the Dead" takes place during World War II. Second, the United States Army is in control of the place where the movie takes place.Answer: [Through reflection to self-correct paths, the answer was found on the revised paths.] The movie "The Naked and the Dead" takes place in Panama. The President of Panama, Juan Carlos Varela, is in control of Panama based on the knowledge triplets.→ → → → → → → ••• ••• → Sub-Objectives: 1.Identify the place where the movie "The Naked and the Dead" takes place; 2. Determine the authority or governing body in control of that place.1. The movie "The Naked and the Dead" takes place in Panama; 2. The President of Panama, Juan Carlos Varela is in control of Panama.Figure 3 : A typical case to compare different methods to answer the complex question. For the convenience of display, we only provide the sub-objective status and partial reasoning paths stored in memory. Topic entities, wrong answers, and correct answers are highlighted in blue, red, and green. The revised path is highlighted with a yellow background. leverages sub-objective status to self-correct the exploration path in the reflection stage and finally provides the correct answer. 
related work: 
 LLM Reasoning. To encourage LLMs to engage in reasoning rather than simply providing answers directly, many researchers instruct LLMs to generate the process of thinking in their outputs [49, 22, 63] . In the early stages, Chain of Thought (CoT) [49] was designed to provide a few examples of intermediate natural language reasoning steps as the prompt. After that, several variants of CoT reasoning with different forms like Tree-of-Thought [54] , Graph-of-Thought [4] , Memory of Thought [24] , and Skeleton-of-Thought [30] were proposed to enhance the thinking process. However, LLMs may make mistakes during the reasoning process. Hence, many works [32, 21, 28, 29] designed self-correction mechanisms based on feedback to rectify flawed reasoning and ensure accuracy. Additionally, large efforts were dedicated to guiding LLMs in understanding complex graph structures [39, 38] and improving their graph reasoning across different graph tasks [11, 10, 9] . However, it is still an open issue to address the outdated knowledge, hallucinations, and opaque decision-making for LLM reasoning. KG-Augmented LLM. Despite the pre-training of LLMs on massive corpora, they still suffer from limitations such as outdated knowledge, hallucinations, and opaque decision-making. An effective approach to address these limitations is to leverage KGs for explicit and editable knowledge provision to LLMs. Previous studies integrated KGs into LLM pre-training [61, 47] or fine-tuning [53, 27] stage, but they merely inject structured knowledge into LLMs' parameters and still leave these limitations unexplored. Therefore, several works [3, 2, 40] first retrieved information from KGs and then directly fed explicit knowledge into LLMs. In this way, LLMs do not involve the graph reasoning process and cannot provide potential insights. Then, a novel KG-augmented LLM paradigm [19, 35] was proposed to treat the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. Although this KG-augmented LLM paradigm has achieved impressive performance, it still faces the challenges of adaptively exploring the KG based on question semantics and self-correcting erroneous reasoning paths. To the best of our knowledge, our work stands out as a pioneering effort in successfully integrating a reflection mechanism for self-correction and adaptive KG exploration into KG-augmented LLMs, effectively enhancing the LLM's reasoning ability. 
conclusion: 
 In this paper, we proposed a novel self-correcting adaptive planning paradigm for KG-augmented LLM named Plan-on-Graph (PoG). To the best of our knowledge, we were the first to incorporate a reflection mechanism for self-correction and adaptive KG exploration into KG-augmented LLMs, effectively augmenting LLM's reasoning ability and efficiency. PoG first decomposed the question into several sub-objectives, and then repeated the process of exploring reasoning paths, updating memory, and reflecting on the need to self-correct reasoning paths until arriving at the answer. To be specific, three important mechanisms were designed to work together to guarantee the adaptive breadth of self-correcting planning for graph reasoning, i.e., Guidance, Memory, and Reflection. Finally, extensive experiments on three real-world KGQA datasets validated not only the effectiveness but also the efficiency of the proposed PoG. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The datasets are from https://github.com/IDEA-FinAI/ToG/tree/ main/data, and we present the licenses in Appendix C. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We will provide the documentation with assets. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The baselines we compare can be categorized into two groups: (1) LLM-only methods; (2) KGaugmented LLM methods, including fine-tuned and prompting methods.• Standard prompting (IO prompt) [6] verifies the ability of LLMs to achieve better performance in task-agnostic, few-shot problems than traditional LMs.• Chain-of-Thought prompting (CoT) [49] generates a series of intermediate reasoning steps in prompts to help LLMs perform better in several NLP tasks.• Self-Consistency (SC) [48] samples multiple, diverse reasoning paths through few-shot CoT, and uses the generations to select the most consistent answer.Finetuned KG-Augmented LLM Methods• UniKGQA [20] unifies the graph retrieval and reasoning process into a single model with LLMs.• TIARA [34] first uses BERT to retrieve a set of schema items, which are further used as the input, together with the question, to T5 for plan generation. They also apply constrained decoding but only for grammaticality.• RE-KBQA [7] capitalizes relations in KGs to enhance entity representations and introduce additional supervision to improve the selection of reasoning paths.• DeCAF [57] combines semantic parsing and LLMs reasoning to jointly generate answers, which also reach salient performance on KGQA tasks.• RoG [27] collaborates LLMs with KGs to achieve trustworthy reasoning to leverage structural information.• RnG-KBQA [55] first uses BERT to rank a set of enumerated candidate programs (up to a limited complexity), and then uses T5 to edit the top programs into more complex programs.• FC-KBQA [58] proposes a fine-to-coarse composition framework to avoid knowledge entanglement and guarantee both generalization ability and logical interpretability.• Pangu [16] considers leveraging the discriminative ability of LLMs. It consists of a symbolic agent with a cooperative neural LLM.• FlexKBQA [26] is a flexible KGQA framework with LLMs. It can utilize a limited set of annotated data to build KGQA for different KGs and query languages.• GAIN [33] pays attention to the robustness of KGQA models. It proposes a data augmentation method to alleviate this problem and further evaluates the distribution shifts including from different aspects.Prompting KG-Augmented LLM Methods• KB-BINDER [23] is developed to challenge the heterogeneity of items from different KGs. It enables few-shot in-context learning over KGQA tasks.• KD-CoT [41] retrieves relevant knowledge from KGs to generate faithful reasoning plans for LLMs.• StructGPT [19] defines the interface of KG data to implement knowledge access and filtering with finite quantity, and leverage the LLM to infer the answer or subsequent planning repeatedly.• Interactive KBQA [52] interacts with KGs directly and then generates logical forms. The interactions are under three designed universal APIs for KGs.• ToG [35] iteratively retrieves relevant triplets from KGs and employs the LLM to assess whether the reasoning paths in beam search are sufficient for answering the question and if further retrieval of the next hop is necessary.In our experiments, we use GPT-3.5 and GPT-4 to serve as the underlying LLMs. We call them by the OpenAI official API 3 . We set the temperature parameter to 0.3, frequency penalty to 0, and presence penalty to 0. The maximum token length for generation is 1024. In all experiments, the depth of exploration is set to 4 to avoid endless exploration. The experiments are conducted on a server with two Intel(R) Xeon(R) CPU E5-2682 v4 @ 2.50GHz and 256 GB RAM memory. Since LLMs are not entirely certain about when to stop, we need to manually set the depth of KG exploration to avoid endless exploration. To investigate the impact of exploration depth on PoG performance, we conduct experiments with depth settings ranging from 1 to 5 on CWQ dataset. As shown in Figure 4 , increasing the depth will improve the performance of PoG. Beyond a depth of 4, the improvement becomes less noticeable. The increase in depth leads to exponential growth in resource and time consumption. Considering the balance between efficiency and effectiveness, we set the depth to 4. In PoG, we design a reflection mechanism to provide the opportunity for self-correction for exploring reasoning paths. In Figure 5 , we calculate the proportion of cases with reverse occurrences among all questions in CWQ, and the results show that 24% of cases involve reversing during the exploration process to achieve self-correction. This demonstrates that LLMs are indeed not always capable of making correct judgments in KG exploration and that self-correction is necessary for KG-augmented LLMs. Figure 6 presents the proportion of correct answers obtained by PoG after self-correction on three datasets. Overall, the self-correction in PoG appears to have positively impacted the accuracy of KGQA, particularly for the WebQSP and CWQ datasets, where the proportion of correct answers reached 64% and 48% after the self-correction process. This analysis suggests that the reflection mechanism in PoG has the potential to enhance the reasoning capabilities of KG-augmented LLM and improve the performance across various datasets by allowing for self-correction and exploration of alternative reasoning paths. Besides, Figure 7 shows another typical case from the testing results on CWQ dataset. We compare the results of PoG, ToG, and CoT in answering the question "What genre of music favored by Claude Debussy appears in the movie Suzanne Farrell: Elusive Muse?". PoG first adaptively identifies the reasoning and planning processes represents an innovative attempt that could further propel research and innovation in broader AI technologies within the artificial intelligence field. (4) Improving User Experience and Expanding Application Domains: With PoG's reasoning capabilities, user experience is greatly improved due to more accurate and quicker responses. Meanwhile, the domains where it can be applied will also expand, particularly in environments that require handling complex queries and responses involving large volumes of data.There are still limitations in using PoG for addressing more complex problems. Some of the key limitations include: (1) Low Self-Confidence: LLMs are still not entirely certain about what information is needed, how many steps are required to extract the information, when to perform dynamic updates, or if the current information is sufficient. In future work, we will focus on the evaluation of LLM's self-confidence. For example, this can be alleviated by training a small model specifically for this evaluation task to improve accuracy. (2) Efficiency: Answering complex questions requires multiple steps. In future work, we aim to design strategies to reduce steps and improve task execution efficiency in situations of high self-confidence. (3) Non-Standardized Query: For less standardized queries, semantic understanding might be insufficient due to limitations in the capabilities of the LLM itself, leading to decreased effectiveness. In future work, we will address this issue by employing SOTA query rewriting methods or interacting with the user to refine the query.https://github.com/liyichen-cly/PoGhttps://huggingface.co/sentence-transformers/msmarco-distilbert-base-tas-bhttps://platform.openai.com/docs/api-reference. 


Decision: In this work, the authors propose Plan-on-Graph (PoG), a self-correcting adaptive planning paradigm, which aim to help KG-augmented LLM systems to solve question answering task. Given a question, the pipeline first decomposes the question into a sequence of sub-objectives; then it repeats the loop of 1) reasoning path exploration in the KG; 2) updating useful information into a memory; and 3) reasoning to find an answer if with sufficient inforamtion or reflect for other reasoning paths otherwise. The authors tested their method on three KGQA tasks, they show PoG outperform a set of baseline systems on all tasks. 

This work has many merits, the all-positive scores from the five reviewers reflect this. All reviewers agree that the work is well motivated as self-correction and adaptive planning seem to enhance LLM system's reasoning capabilities; applying such idea and technique in KG path exploration for KGQA is novel; the presentation is clear, the paper is well structured, especially the related work section; the experiment design is good, experimental results support the claim well; the fact that the method is training-free makes it easy to plug-and-play. 

On the negative side, reviewers' main concerns were around the following aspects:
- Unclear how PoG connects/compares with prior works that also utilize graph-structured memory, such as Graph-of-Thought. 
- Unclear about efficiency, latency and cost limitation because PoG (as a inference-only method) may require huge amount of API calls when the question is compositional complex or the KG is huge. 
- Unclear about how PoG could be applied using more accessible models as backbone.
- Missing implementation details.

Reviewers agree that in general the authors did a good job addressing and clarifying the concerns, two reviewers have increased their score from 4 to 5. However, there are a few points remain less satisfied:
- The authors suggest that their proposed paradigm is model-agnostic, their goal is to enhance the reasoning capabilities of LLMs, rather than simply improving the performance of KGQA. This is only partially true, because there lacks of evidence the PoG could be directly applied to weaker/open-sourced LLM systems. What is the minimal requirements on the backbone end to make this working?
- In multiple steps of the pipeline, the LLM is asked to generate an output based on the provided information (e.g., retrieved from the KG) and `its own knowledge`. The fail of disentangling external knowledge and the LLM's embedded knowledge makes it less straightforward to understand the proposed method.
- It is only in the appendix that the authors say they are using in-context learning in all the prompts (Appendix A). Specifically, they use 5-shots learning on all datasets (Appendix E). They fail to provide any detail on how the five examples are being selected/retrieved, are they using the same in-context examples across datasets, across data points, or per data point? This makes it even more difficult to fully understand how the proposed method works. 
- Related to the above points, although the authors provide a good ablation study showing how each components could affect the overall system performance, they fail to discuss deeper how different design of each component could make a difference. Because PoG is a training-free method, specific prompt design, including the way to obtain the in-context examples could be rather important. This once again is related to how accessible the method is, because different backbone model may need very different prompt designing/engineering practice.

Given the all-positive scores, I tentatively recommend accept, but because:
- there are still some critical points unclear (see my review above); and
- all reviewers end up giving lukewarm scores,

I will not fight for acceptance if SAC suggests otherwise.

Reviews: review: paper_summary: paper_summary: This paper introduces Plan-on-Graph (PoG), a novel self-correcting adaptive planning paradigm for Knowledge Graph-augmented Large Language Models (KG-LLM). PoG aims to address limitations in existing KG-augmented LLM approaches by decomposing questions into sub-objectives and iteratively exploring reasoning paths, updating memory, and reflecting on the need for self-correction. The method incorporates three key mechanisms: Guidance, Memory, and Reflection, designed to ensure adaptive breadth in self-correcting planning for graph reasoning. Experiments on three real-world datasets demonstrate the effectiveness and efficiency of the proposed approach.
strength_weakness: strength_weakness: - Missing important implementation details: While the paper provides visualized knowledge subgraphs to represent path exploration and memory updates, there is no explicit template or example showing how this graph or tree structure is exactly flattened in the prompts of the LLM agent. The use of arrows for paths in both Figure 3 and 5 creates ambiguity about the real implementation. This makes reader unclear and concerns about the real implementation. 

- Concerns about model dependency: The proposed PoG method is a pure prompting approach, which raises questions about its applicability and performance on smaller open-source models. There are concerns about whether the performance improvements achieved by PoG are largely due to the capabilities of the advanced base models used, rather than the method itself. 


- Lack of critical evaluations: The paper is missing several important analyses, such as:
  - The proportion of cases where the LLM needs to trigger reverse direction path visiting
  - The frequency with which this revisiting leads to correct or incorrect answers
  - How the number of breadth in each step affects the effectiveness of the proposed method
  - Whether the length of hops from the query entity to the answer entity affects the number of revisiting and the method's effectiveness
  - How consistent or diverse is the exploration path under the proposed method if experiment with multiple trails?

- Minor thoughts: Additionally, it would be interesting if the authors could discuss or explore whether this method of constructing paths and memories could support effective post-training of open-source models, though this is not a critical weakness.
questions: questions: Please refer to the questions raised in Weakness section.
limitations: limitations: Yes, the author has discussed the limitations in Appendix G, and leave them as future work.
paper_summary: paper_summary: The paper introduces Plan-on-Graph (PoG), a new paradigm for integrating LLMs with KGs to enhance their reasoning capabilities. The main innovation lies in PoG’s self-correcting adaptive planning mechanism, which addresses the limitations of existing KG-augmented LLMs that rely on predefined exploration spaces and unidirectional navigation. PoG breaks down complex questions into sub-objectives, then iteratively explores and corrects reasoning paths using a combination of Guidance, Memory, and Reflection mechanisms. The Guidance mechanism helps in decomposing the question, Memory stores historical data to support reasoning, and Reflection allows the model to self-correct erroneous paths. The authors validate PoG’s effectiveness through extensive experiments on three KGQA datasets, showing that it outperforms state-of-the-art methods in terms of both accuracy and efficiency.
strength_weakness: strength_weakness: - The PoG framework’s complexity, with its multiple components and iterative processes, might make it challenging to implement and optimize.

- The necessity for extensive prompt engineering and management of memory and reflection mechanisms could be resource-intensive.

- While PoG performs well on the evaluated datasets, its generalization to other domains or types of KGs is not thoroughly explored.

- Not compared with other RAG methods such as REPLUG[1]. 

- The efficiency and cost can be an important limitation. Efficiency in terms of computational resources and time is a concern, as PoG, while efficient, is still resource-intensive. The reliance on KGs means that the model’s knowledge can become outdated if the KG is not regularly updated.




[1] REPLUG: Retrieval-Augmented Black-Box Language Models
questions: questions: - How does the choice of the exploration depth (set to 4) impact the performance across different datasets? Have you experimented with varying this parameter?

- How does PoG perform with larger and more complex KGs that were not part of the evaluation datasets?

- How does PoG compare with other recent advancements in RAG methods?

- How do you decide when to use the PoG and when to use regular LLM call? Given the expensive cost of PoG, I think it is very unrealistic to run all user queries with PoG. I'd suggest the authors discuss how PoG can be applied in real-world application scenarios.
limitations: limitations: discussed
paper_summary: paper_summary: This paper proposes a self-correcting adaptive planning paradigm for KG-augmented LLM called PoG. It consists of four components: task decomposition, path exploration, memory updating, and evaluation. Experiments on three datasets demonstrate the effectiveness of PoG, outperforming previous methods.
strength_weakness: strength_weakness: 1.	Although the paper has achieved certain results in the experiments, it lacks sufficient analysis on why PoG can achieve better results compared to the baseline models. Additionally, for other KG-augmented LLMs such as ToG[1], ARI[2], and RoG[3], the paper does not sufficiently highlight the advantages and differences of the proposed method.
2.	There is a concern about the efficiency of the proposed method. In a KG, there may be thousands or even millions of entities. Taking Figure 2 as an example, a singer may have hundreds of songs. When the LLM completes the first step of reasoning (finding the singer's songs) and proceeds to the second step of finding the songs that have won awards, the LLM may need to be called hundreds of times? This could potentially lead to inefficiency and increased computational costs. 
3.	The paper could provide more details on the potential limitations and challenges of PoG, as well as possible strategies to overcome them.

[1] Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph
[2] Temporal knowledge question answering via abstract reasoning induction
[3] Reasoning on graphs: Faithful and interpretable large language model reasoning
questions: questions: 1.How are the topic entities in the question obtained? 
2. When there are too many candidate entities, how does the model ensure the efficiency of reasoning? 
3. After selecting the wrong relation or entity, how does the model perform backtracking and reflection?
limitations: limitations: The paper could provide more details on the potential limitations and challenges of PoG.
paper_summary: paper_summary: This paper proposes a new self-correcting adaptive planning paradigm for KG-augmented LLM named Plan-on-Graph (PoG). It has three important mechanism: Guidance, Memory and Reflection. Experiments on three knowledge graph question answering datasets show good results.
strength_weakness: strength_weakness: The weakness of this paper is from the novelty side. The proposed method PoG is like a trivial improvement on the knowledge graph reasoning task based on [Graph of Thoughs (GoT)](https://ojs.aaai.org/index.php/AAAI/article/view/29720). The path explored in this paper also forms a graph like the graph in GoT paper. And the self-reflection and evaluation are like the backtracing and refining steps in GoT paper. The proposed method just adapts these GoT steps in knowledge graph reasoning task.
questions: questions: Can the authors explain more about the difference of the proposed PoG method and GoT method? My suggestion is to clarify more on why PoG is not a trivial adaption of GoT on knowledge graph tasks.
limitations: limitations: Yes  the authors adequately addressed the limitations.
paper_summary: paper_summary: This paper aims to improve KG-augmented LLMs by introducing a self-correcting adaptive planning paradigm. PoG uses three important mechanisms: Guidance, Memory, and Reflection. Guidance leverages LLM to decompose the query into subqueries; Memory stores historical retrieval and reasoning information for reflection; and Reflection uses LLM to decide whether to continue exploration or generate the final output. Experiments show the effectiveness of PoG compared to other SOTA methods and also demonstrate improved efficiency.
strength_weakness: strength_weakness: 1. The self-correction is restricted to the retrieval process. However, whether the final answer can be correctedly answer also depend on how the query was decomposed. If the decomposed sub-queries are not suitable for retrieval, the self-correction cannot handle this case.

2. Even though PoG improves efficiency compared with previous methods, it still requires multiple interactions with the LLM (more than 6) and incurs latency, especially for real-time QA (more than 10 seconds on all datasets as reported).
questions: questions: 1. In Table 4, is the number of input/output tokens the average of every single LLM call or for the whole QA?
limitations: limitations: See weakness above

Comments: title: Official Comment by Reviewer vd15
Response
Response
comment: Thank you very much for acknowledging our paper! We will carefully incorporate these clarifications and further improve the quality of our paper.
Thank you for your response and clarification. I have also read the comments from other reviewers and authors' response. I'd like to keep my original rating.
Thank you very much for acknowledging our paper! We will carefully incorporate these clarifications and further improve the quality of our paper.
Dear authors,

Thanks for your answer. Now I understand more about the innovation part of this paper. I will keep the current score.

Thanks,

Reviewer vd15
Thank you very much for your feedback and for acknowledging our work. We appreciate your suggestions and will definitely consider adding more details to further enhance our paper in future editions. We are grateful for the score increase!
Thanks the authors for the rebuttal, especially the additional prompting details and analytical evaluations well address my original concern, please consider adding these important details into future editions of the paper. Regarding the LLM dependency and potential discussion of post-training, I acknowledge and understand the proposed method is training-free as this is a pure prompting approach. My original concern is raised based on the relatively lower reasoning and instruction-following ability provided by most open-sourced smaller LLMs, it would be definitely beneficial if the proposed method could exhibit enhancement on these models. Overall, I appreciate the work and the responses, I have raised my score. Thanks.
Dear Reviewer kA7J,

We would like to express our sincere gratitude for the time and effort you spend reviewing our paper. As **the author/reviewer discussion stage draws to a close**, we are eager for your response to ascertain if our detailed response has sufficiently addressed your concerns. We would be honored to address any further questions you may have. *We eagerly anticipate and highly value your re-evaluation of our paper.* 

Thank you once again for your thorough review of our paper.

Best regards,

Authors of Submission 4240
We greatly appreciate your positive comments and acknowledgment of our paper! In the final version, we will carefully incorporate these clarifications.
Thank you to the authors for addressing my questions. I will maintain my positive score.
Thank you very much for acknowledging our paper! We will carefully incorporate these clarifications and further improve the quality of our paper.
The author's response partially addressed my concerns. I have raised my score. I hope the author further revises the manuscript to make it more complete.