Chain-of-Question: A Progressive Question Decomposition Approach for Complex Knowledge Base Question Answering (2024)

Abstract: Complex KBQA leverages the knowledge base (KB) to answer complex natural questions involving complicated semantics like multi-hop reasoning. Existing methods involve a question decomposition process, i.e., breaking a complex question into several simpler sub-questions, to assist obtaining logical forms for querying the KB. However, existing question decomposition process derives all sub-questions directly according to the original question, resulting in limitations when one sub-question relies on the answer from a previous one. In this work, we propose Chain-of-Question, a progressive question decomposition approach to address complex KBQA challenges. First, inspired by chain-of-thought, we design a prompt to guide LLM to sequentially decompose multiple semantically clear sub-questions and provide corresponding reference answers, where each step of the decomposition relies on the previous results. Next, we utilize the decomposition result to select relevant patterns (relation-entity pairs) as accurate and faithful auxiliary information for the following logical form generation. Finally, we jointly perform logical form generation and answer prediction, utilizing the predicted answer to supplement non-executable logical forms. Experimental results demonstrate that our method achieves state-of-the-art performance on multiple datasets.

## introduction knowledge base question answering (kbqa) which empowers machines to answer natural questions using the knowledge base (kb) is a critical task in developing high-confidence ai systems. semantic parsing, as a prevailing approach, transforms natural questions into logical forms with robust interpretability and executes them on the kb to generate answers, achieving notable performance @xcite @xcite et al., 2022). however, addressing complex kbqa, which involves answering complex questions containing complicated semantics such as multi-hop reasoning, remains a challenge. for machines, comprehending complex questions directly and obtaining accurate and complete logical forms can often be difficult, consequently leading to the nonexecutable problem. a natural approach to comprehend complex question is question decomposition, the core idea of which is to break down complicated semantics into simpler semantic components. therefore, many studies utilize question decomposition to assist semantic parsing, which involves question decomposition process to understand the question and then integrate the decomposition result as auxiliary information for obtaining the logical form. in some prior methods @xcite , a question is decomposed into two subquestions, where the relationship between these two sub-questions could be coordinate (connected using conjunctive descriptions) or inclusion (one is included in the other). however, they cannot handle complex questions that involve more semantic components and types of relationships. @xcite proposed question decomposition tree to better model the semantic structure of questions, they linearize the question decomposition tree into a question with inserted special tokens and directly generate the linearized question decomposition tree as the decomposition result, enabling the representation of questions with multiple semantic components that have different relationships. ## method in this section, we elaborate on our method coq for addressing complex kbqa problems. figure 2 provides an overview of our approach, which consists of three main components: chain of question decomposition construction, relevant schema items retrieval and patterns selection, and performing joint logical form generation and answer prediction via multitask learning. we will provide the details in the following subsections. ## error analysis we conduct an analysis on 200 randomly select questions from the cwq test set for which coq does not answer correctly (@xmath0.0). the errors can be summarized as follows: schema items error (34.5%) the largest proportion of incorrect answers is attributed to errors in generating schema items, i.e., relations and entities, in the logical forms. despite providing relevant patterns as auxiliary information to improve accuracy and faithfulness, the retrieval of schema items remains a significant challenge for complex kbqa. logical form structure error (25.5%) despite coq explicitly modeling the question structure through question decomposition and enhancing the accuracy of logical form structure, there are still some complex questions for which it cannot generate accurate logical form structures. the reason could be that coq does not enforce the decomposition results to fully match particularly these complex structures. answer prediction error (28.3%) the accuracy of the answer prediction task still has limitations, which results in some questions remaining not being correctly answered. ## conclusion this paper introduces chain-of-question (coq), a progressive question decomposition method for addressing complex kbqa. we first design a prompt that guides a llm to progressively construct a chain of question decomposition, containing subquestions and corresponding reference answers. relevant schema items are then retrieved, and patterns are selected to improve the accuracy and faithfulness of logical forms. the final step involves jointly generating logical forms and predicting answers with the assistance of the chain of question decomposition and selected patterns, using predicted answers to mitigate non-executable issues. experimental results demonstrate the method's state-of-the-art performance on multiple datasets. the impact of schema item retrieval on final predictions is also explored, revealing significant potential. future research will focus on refining schema item accuracy and effectively modeling the semantic structure of questions.