Following Occam’s Razor: Dynamic Combination of Structured Knowledge for Multi-Hop Question Answering using LLM s (2025)

Abstract: Multi-hop question answering is a challenging task that requires capturing information from different positions in multiple documents. Recently, several methods propose to enhance Large Language Models (LLMs) by incorporating structured knowledge, aiming to grasp key information for solving this task. Despite certain achievements, they still face the following challenges: 1) The neglect of text-based reasoning capabilities. 2) Information redundancy between text and triples. 3) Information loss during structured knowledge extraction. To solve the above challenges, in this paper, we propose Dynamic Combination of Structured Knowledge (DCSK), a novel framework for integrating text-based and triple-based paradigms. Following Occam’s Razor, DCSK dynamically determine the necessity of structured knowledge by the designed multi-faceted evaluation, which systematically assess the correctness, clarity, and informativeness of text-based prediction. For questions that require structured knowledge, we develop an iterative fact refiner that screens for question-relevant triples, verifies their factual adequacy, and thereby effectively excludes irrelevant and redundant information. Furthermore, based on the verification, we construct an adaptive knowledge reasoner that dynamically adjusts the need for text supplementation, thus mitigating the information deficiency in selected triples. Extensive experiments on three MHQA datasets demonstrate the efficiency and effectiveness of DCSK.

## introduction multi-hop question answering (mhqa) is a challenging task in the field of natural language processing, which needs to capture relevant information from multiple locations of the documents for reasoning @xcite . for example, to answer "arthur marryatt is a member of a sports association that has whom as president?", it is essential to first identify the specific sports association to which "arthur marryatt" belongs, and subsequently ascertain the identity of its president. this process necessitates a certain level of combinatorial reasoning capability @xcite . as large language models (llms) have demonstrated excellent text understanding and reasoning capabilities @xcite @xcite , they have proven to be indispensable in the mhqa task @xcite . recently, several studies propose to extract structured knowledge from documents by constructing triples @xcite @xcite . this triple-based paradigm aims to assist llms in comprehending the relationships between various entities and facilitating more reliable and interpretable reasoning. although the certain progress made, it still faces the following challenges: (1) the neglect of textbased reasoning capabilities. previous work assume that all questions must be addressed by integrating multi-step processed triples, which undoubtedly necessitates substantial computing resources. however, even when only the original unstructured text is available, llms possess sufficient basic reasoning capabilities to accurately answer certain questions, especially with the assistance of some plugin methods (such as chain of thought (cot) @xcite ). as shown in figure 1 (a) , the majority of questions can be accurately addressed by both paradigms. in accordance with occam's razor, this subset of data does not necessitate additional complex processing. furthermore, it is evident that a considerable number of problems remain solvable by only one paradigm. this indicates that if the two paradigms are appropriately integrated, it is possible to not only conserve unnecessary computing resources but also enhance performance to some extent. to address the above challenges, we propose dynamic combination of structured knowledge (dcsk), a novel framework for integrating textbased and triple-based paradigms that both increases efficiency and the reasoning capabilities of the llms. specifically, we initially employ the foundational reasoning capabilities of llms for text-based inference. subsequently, we leverage a multi-faceted evaluation to systematically assess the correctness, clarity, and informativeness of the predictions. through this mechanism, we are able to identify the questions that genuinely necessitate the structured knowledge. within the triplebased paradigm, given that answering questions requires only core and critical steps, we design an iterative fact refiner to eliminate irrelevant and redundant information. this process initially selects triples relevant to the question and subsequently verifies the factual adequacy of the selected triples to ascertain whether they provide sufficient information. if verification fails, the process repeats until it passes or the predefined maximum number of attempts is reached. finally, we construct an adaptive knowledge reasoner, that dynamically determines whether to incorporate the original text as supplementary input based on the verification results, thereby mitigating the issue of information insufficiency in selected triples. our main contributions are as follows: ## conclusion in this paper, we proposed dynamic combination of structured knowledge (dcsk), a novel framework for integrating text-based and triple-based paradigms to address the multi-hop question answering. dcsk dynamically determined whether the question requires the intervention of structured knowledge through the multi-faceted evaluation, which assess the correctness, clarity, and informativeness of text-based prediction. for questions that require intervention, we designed an iterative fact refiner to select and verify the triples related to the question, thereby excluding redundant information. additionally, based on the verification, we constructed an adaptive knowledge reasoner to adjust the need for text supplementation to address the information deficiency in triples. extensive experiments demonstrated the effectiveness of dcsk.