From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models (2023)

Abstract: Reasoning is a distinctive human capacity, enabling us to address complex problems by breaking them down into a series of manageable cognitive steps. Yet, complex logical reasoning is still cumbersome for language models. Based on the dual process theory in cognitive science, we are the first to unravel the cognitive reasoning abilities of language models. Our framework employs an iterative methodology to construct a Cognitive Tree (CogTree). The root node of this tree represents the initial query, while the leaf nodes consist of straightforward questions that can be answered directly. This construction involves two main components: the implicit extraction module (referred to as the intuitive system) and the explicit reasoning module (referred to as the reflective system). The intuitive system rapidly generates multiple responses by utilizing in-context examples, while the reflective system scores these responses using comparative learning. The scores guide the intuitive system in its subsequent generation step.Our experimental results on two popular and challenging reasoning tasks indicate that it is possible to achieve a performance level comparable to that of GPT-3.5 (with 175B parameters), using a significantly smaller language model that contains fewer parameters (<=7B) than 5% of GPT-3.5.

## introduction the human brain is akin to a garden, where instincts are seeds that sprout and grow, while reason acts as the gardener, pruning and nurturing the plants of knowledge to bloom into the flowers of enlightenment. for machines, recently, large language models (llms) have demonstrated their abilities to tackle diverse tasks through instantaneous question answering, exhibiting some levels of intelligence @xcite @xcite . however, to cross the chasm between machines and humans, three main challenges still lie ahead: 1) reasoning ability. when it comes to mathematical and reasoning problems, the performance of lms is still not satisfactory @xcite . 2) cognition capacity. the evaluation and decision-making process regarding the problem and its current state is of paramount importance, especially when dealing with problems that involve lengthy reasoning chains or multi-step solutions. however, current methods @xcite often lack comprehensive validation and tend to focus on verifying intermediate results @xcite . 3) efficiency. the deployment and inference costs of llms are relatively high, especially when utilizing parameterfree inference enhancement techniques @xcite . these techniques require extensive contexts and multiple steps of answer generation, leading to a further increase in inference costs and time. we suggest that valuable insights into addressing these challenges can be derived from the cognitive processes of humans. in cognitive science, the dual process theory @xcite @xcite states that our brain initially employs an implicit, unconscious, and intuitive process known as the intuitive system, which retrieves relevant information. this is followed by an explicit, conscious, and controllable reasoning process called the reflective system. the intuitive system is capable of providing resources in response to requests, while the reflective system facilitates a deeper exploration of relational information through sequential thinking in the working memory. although slower, the reflective system possesses a unique human rationality @xcite . in complex reasoning tasks (including logical reasoning and mathematical reasoning tasks), cogtree. an intuitive system is employed to generate candidate plans, while a reflective system verifies the plausibility of each plan to guide the next generation of the intuitive system. this iterative process is repeated to create the tree structure for reasoning. ## cognitive tree (cogtree) framework the reasoning ability of humans primarily arises from acquiring pertinent information from the environment and subconscious processing @xcite . in our approach, we incorporate a tree structure to systematically tackle reasoning problems, taking inspiration from human problem-solving procedures. this methodology aligns with the planning processes analyzed by @xcite . newell and his colleagues defined problem solving @xcite as the exploration of a combinatorial problem space, represented as a tree. in our mathematical and logical reasoning setting, each node n in the cognitive tree t represents either a theory in a logical set or the solution to a sub-problem in a mathematical question. an edge e of the tree corresponds to the evaluation of the current node's state s, which can be a confidence score or a classification result. the problem decomposition module (i.e., the intuitive system) receives the theory and the original problem as input, and generates the decomposition of the original problem. next, the newly generated nodes are used to expand the tree, providing the reasoning module (i.e., the reflective system) with the information that needs to be verified. the discernment capacity of the reflective system plays a pivotal role in enhancing the overall efficacy of the model @xcite . in particular, we utilize the cross-checking technique to not only verify the precision of intermediate outcomes but also validate the accuracy of the entire reasoning process upon its completion. however, relying solely on individual decision judgments for training the model is inadequate @xcite . to augment the model's ability to evaluate the state s, we propose the implementation of a comparative reinforcement approach. this approach entails introducing a new training objective, whereby the model is tasked with maximizing the disparity in vector space between representations of correct decisions and representations of incorrect or ambiguous decisions. in addition, efficiency pertains to the efficacy of the problem decomposition and verification process. generating lengthy texts using a model as massive as 175b entails substantial time and financial expenses. to tackle this issue, our method can be implemented by simply fine-tuning a comparatively smaller model (1.5b or 7b) exclusively for reasoning tasks. this enables us to deploy the model for these tasks with minimal costs. ## implementation in this section, we describe the implementations of cogtree in detail. ## experiments we perform an extensive evaluation of our method utilizing two well-established benchmark datasets: the entailment bank (eb) @xcite and gsm8k @xcite . eb consists of human-annotated tuples containing information about theories, provable goals, and corresponding reasoning paths. gsm8k, on the other hand, presents a challenging arithmetic reasoning task that language models frequently find difficult to tackle @xcite . examples of these two datasets are in ## related work multi-step reasoning. reasoning has long been a key focus in natural language processing research. initially, most studies concentrated on basic tasks like single-sentence language inference @xcite @xcite @xcite and commonsense inference @xcite @xcite in a single step. lately, there has been a surge in research interest regarding multi-step reasoning and mathematical problem solving. the search space for a correct reasoning path is extremely vast and complex. previous approaches @xcite @xcite predominantly emphasized a bottom-up reasoning approach, with a strong focus on system design. in contrast, our approach to constructing the cognitive tree adopts a top-down methodology, reducing the burden during the era of generative models. this top-down approach offers greater flexibility for application across models of varying scales. llm as evaluation. the utilization of language models (llms) to evaluate the validity of their own predictions is gaining significance as a procedure in problem-solving. the introduction of the self-reflection mechanism by @xcite @xcite involves lms providing feedback to their generated candidates. tree of thought @xcite and our approach share a common utilization of a tree-based structure for problem-solving. however, tot primarily concentrates on tree construction and limited selfvalidation of intermediate results. according to the dual process theory, which suggests that validation requires deeper levels of thinking, our approach incorporates a contrastive learning method to enhance the model's ability to distinguish accurate results and facilitate comprehensive global validation of the generated outcomes. ## conclusion in this paper, we proposed a new framework named cogtree to address complex logical reasoning and mathematical problems. the process of reasoning involves constructing a cognitive tree, where the nodes represent the decomposition of complex problems into sub-problems, and the edges represent judgments regarding the correctness of the decomposition. based on the implementation of our approach over gpt2 and llama, we have achieved results comparable to gpt-3.5 on eb and gsm8k datasets, indicating the effectiveness of our framework.