C o TKR : Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering (2024)

Abstract: Recent studies have explored the use of Large Language Models (LLMs) with Retrieval Augmented Generation (RAG) for Knowledge Graph Question Answering (KGQA). They typically require rewriting retrieved subgraphs into natural language formats comprehensible to LLMs. However, when tackling complex questions, the knowledge rewritten by existing methods may include irrelevant information, omit crucial details, or fail to align with the question’s semantics. To address them, we propose a novel rewriting method CoTKR, Chain- of-Thought Enhanced Knowledge Rewriting, for generating reasoning traces and corresponding knowledge in an interleaved manner, thereby mitigating the limitations of single-step knowledge rewriting. Additionally, to bridge the preference gap between the knowledge rewriter and the question answering (QA) model, we propose a training strategy PAQAF, Preference Alignment from Question Answering Feedback, for leveraging feedback from the QA model to further optimize the knowledge rewriter. We conduct experiments using various LLMs across several KGQA benchmarks. Experimental results demonstrate that, compared with previous knowledge rewriting methods, CoTKR generates the most beneficial knowledge representation for QA models, which significantly improves the performance of LLMs in KGQA.

## introduction large language models (llms) have achieved remarkable performance across various natural language processing tasks, marking a significant milestone @xcite @xcite @xcite . despite their superior performance in zero-shot scenarios @xcite , they still encounter factual errors, known as "hallucinations" @xcite , especially in knowledgeintensive tasks like question answering (qa) @xcite @xcite . this issue arises due to the intrinsic limitations of llms, including factual inaccuracies and outdated knowledge @xcite . to address this challenge, a substantial of work @xcite @xcite retrieves task-relevant knowledge from external sources as context, thereby enhancing the capabilities of llms in downstream tasks, known as retrieval-augmented generation (rag) @xcite @xcite . recent work @xcite @xcite under the rag paradigm explores the use of knowledge graphs (kgs) @xcite as an information source to enhance the capabilities of llms in question answering (qa). unlike typical qa tasks, a key challenge in kgqa under this paradigm lies in transforming question-related subgraphs into natural language that llms can understand while preserving the structural information @xcite @xcite . this process is referred to as knowledge rewriting (kr) in this study. as illustrated in figure 1 , this paper summarizes the commonly used knowledge rewriting methods in existing work. most prior studies @xcite @xcite employ simple linear concatenation method (triple), which concatenates the subject, relation, and object of each triple to form triple-form text. additionally, considering that llms are pretrained on text corpora and struggle with structured triple-form text, some efforts @xcite @xcite focus on converting triples into natural language through kg-to-text. furthermore, given that retrieved subgraphs often contain redundant information irrelevant to the question, other studies @xcite ; dernbach et al., 2024) aim to extract question-relevant knowledge from the triples to generate summary pertinent to the question. although these strategies are effective, they exhibit several limitations: (1) redundancy or omissions. as illustrated in figure 1 , knowledge generated by triple and kg-to-text are verbose, containing excessive irrelevant information. summary provides a question-related summary but attempts to organize all relevant knowledge in one step. given the extensive knowledge necessary to address complex questions, this method may not encapsulate all critical information, potentially resulting in the omission of key points. (2) semantic mismatch. the three existing methods shown in figure 1 ignore the semantics of the question and lack a logical organization that aligns with the question's reasoning path. to this end, we propose chain-of-thought enhanced knowledge rewriting, cotkr. inspired by react @xcite , the core of our method involves generating reasoning traces and corresponding knowledge in an interleaved manner. as shown in figure 1 , we alternate the following two operations: (1) reasoning: decomposing the question to identify the knowledge required for inference; (2) summarization: summarizing the relevant knowledge from retrieved triples, informed by the reasoning step's output. by integrating chain-of-thought (cot) @xcite with knowledge rewriting, cotkr filters out irrelevant information and extracts question-related knowledge. moreover, it generates a well-organized knowledge representation foot_0 semantically aligned with the question. unlike traditional cot applications in qa, our framework employs the knowl-edge rewriter to first summarize knowledge, which then serves as contextual information to enhance qa performance. this strategy offers superior robustness. although the summary might be inaccurate, it still contributes valuable information, potentially leading to correct answers. however, applying cot to qa requires more precise reasoning chains, which are significantly affected by the error propagation @xcite . to train knowledge rewriters based on llms, we design a training framework for cotkr. in the first stage, inspired by previous work @xcite @xcite , we use knowledge representations generated by chatgpt to guide the supervised fine-tuning of the knowledge rewriter, enabling it to initially master the capability of knowledge rewriting. in the second stage, we introduce preference alignment from question answering feedback (paqaf) to bridge the preference gap between the knowledge rewriter and the qa model. this method evaluates the quality of different knowledge representations based on the corresponding responses from the qa model. subsequently, it constructs preference pairs, and fine-tunes llms through direct preference optimization (dpo) @xcite . we conduct experiments on grailqa @xcite and graphquestions @xcite , comparing commonly used knowledge rewriting methods in existing work. contrary to previous findings @xcite , which suggest that llms perform better with knowledge in tripleform rather than in natural language, our findings demonstrate that llms can significantly benefit from knowledge represented in carefully crafted natural language. this indicates that our method could substantially enhance the performance of llms in kgqa. the main contributions of this paper are: ## conclusion in this paper, we propose chain-of-thought enhanced knowledge rewriting, cotkr, for higher quality knowledge representation of triples in kg augmented qa. to bridge the preference gap between the knowledge rewriter and the qa model, we propose preference alignment from question answering feedback, paqaf. experimental results demonstrate that, compared with existing knowledge rewriting methods, cotkr can generate the most beneficial knowledge representation for qa models. in future work, we will go beyond kgqa to explore knowledge representations for other kinds of structured data for rag.