Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization (2024)

Abstract: Despite the advances in large language models (LLMs), how they use their knowledge for reasoning is not yet well understood.In this study, we propose a method that deconstructs complex real-world questions into a graph, representing each question as a node with predecessors of background knowledge needed to solve the question. We develop the DepthQA dataset, deconstructing questions into three depths: (i) recalling conceptual knowledge, (ii) applying procedural knowledge, and (iii) analyzing strategic knowledge. Based on a hierarchical graph, we quantify forward discrepancy, a discrepancy in LLM performance on simpler sub-problems versus complex questions. We also measure backward discrepancy where LLMs answer complex questions but struggle with simpler ones. Our analysis shows that smaller models exhibit more discrepancies than larger models. Distinct patterns of discrepancies are observed across model capacity and possibility of training data memorization. Additionally, guiding models from simpler to complex questions through multi-turn interactions improves performance across model sizes, highlighting the importance of structured intermediate steps in knowledge reasoning. This work enhances our understanding of LLM reasoning and suggests ways to improve their problem-solving abilities.

## introduction with the rapid advancement of large language models (llms), research interest has increasingly centered on their reasoning capabilities, particularly in solving complex questions. while many studies have assessed the general reasoning capabilities of llms @xcite @xcite , the specific aspect of how these models recall and then utilize factual knowl-* equal contribution. ## related work recent advancements have highlighted the impressive reasoning abilities of transformer language models across a wide range of tasks @xcite . despite the success, numerous studies have found that these models often struggle with various types of reasoning, such as commonsense and logical reasoning @xcite . even advanced models like gpt-4 @xcite have been noted to struggle with implicit reasoning over their internal knowledge, especially when it comes to effectively combining multiple steps to solve compositionally complex problems @xcite @xcite @xcite . in our proposed dataset, the most complex questions often necessitate implicit intermediate steps to reach a conclusion, which can be benefited from explicit verbalized reasoning. however, unlike previous works, our setup does not induce detailed step-by-step explanation contained in an answer to a complex question. instead, we represent intermediate steps for a complex question in the form of sub-questions and gather answers to every subquestion, testing a model's understanding of intermediate knowledge individually. our approach is similar to strategic question answering with intermediate answers @xcite , but we further ensure a hierarchy of decompositions based on knowledge complexity. this allows examining discrepancies between questions of varying complexities, providing a distinct assessment of multi-step reasoning abilities. another line of work focuses on understanding transformers' knowledge and reasoning through controlled experiments @xcite @xcite @xcite . numerous studies on implicit reasoning often aim to identify latent reasoning pathways, but most have focused on simple synthetic tasks or toy models @xcite @xcite , or evaluating through binary accuracy of short-form model predictions without considering intermediate steps @xcite . our depthqa, in contrast, challenges a model to answer complex real-world questions that require diverse reasoning types in long-form text. depthqa further requires diverse types of reasoning across different depths, such as inductive and procedural reasoning, in addition to the comparative and compositional reasoning explored in prior studies @xcite @xcite . this approach provides a more practical and nuanced assessment of the model's reasoning capabilities. ## graph-based reasoning framework we develop a novel graph-based representation that delineates the dependencies between different levels of knowledge. we represent nodes as questions (section 3.1) and edges as reasoning processes (section 3.2). based on the graph definition, we construct a dataset that encompasses diverse concepts and reasoning types (section 3.3). ## experiments in this section, we present experiments on the depthwise reasoning ability of llms using depthqa. we first explain the evaluation metrics and models (section 4.1). experimental results that follow are overall depthwise and discrepancy evaluation results (section 4.2), the impact of memorization in knowledge reasoning (section 4.3), and the effect of enforcing knowledge-enhanced reasoning via multi-turn inputs or prompt inputs (section 4.5). ## conclusion in this study, we explore the reasoning capabilities of llms by deconstructing real-world questions into a graph. we introduce depthqa, a set of deconstructed d 3 questions mapped into a hierarchical graph, requiring utilization of muliple layers of knowledge in the sequence of d 1 , d 2 to d 3 . this hierarchical approach provides a comprehensive assessment of llm performance by measuring forward and backward discrepancies between simpler and complex questions. our comparative analysis of llms with different capacities reveals an inverse relationship between model capacities and discrepancies. memorization analysis suggests that the sources of forward and backward discrepancies in large models stem from different types of failures. lastly, we demonstrate that guiding models from shallower to deeper questions through multiturn interactions stabilizes performance across the majority of models. these findings emphasize the importance of intermediate knowledge extraction in understanding llm reasoning capabilities.