Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems (2024)

Abstract: Human cognition exhibits systematic compositionality, the algebraic ability to generate infinite novel combinations from finite learned components, which is the key to understanding and reasoning about complex logic. In this work, we investigate the compositionality of large language models (LLMs) in mathematical reasoning. Specifically, we construct a new dataset MathTrap by introducing carefully designed logical traps into the problem descriptions of MATH and GSM8K. Since problems with logical flaws are quite rare in the real world, these represent “unseen” cases to LLMs. Solving these requires the models to systematically compose (1) the mathematical knowledge involved in the original problems with (2) knowledge related to the introduced traps. Our experiments show that while LLMs possess both components of requisite knowledge, they do not spontaneously combine them to handle these novel cases. We explore several methods to mitigate this deficiency, such as natural language prompts, few-shot demonstrations, and fine-tuning. We find that LLMs’ performance can be improved through the above external intervention. Overall, systematic compositionality remains an open challenge for large language models.

## introduction humans excel at learning fundamental concepts and skills, systematically combining them to solve new problems. for instance, when a person possesses (a) the knowledge of how to solve quadratic equations with one variable, and (b) the understanding of what integers are, they can combine these two domains of knowledge to tackle the problem "find the integer solutions of x 2 +@xmath0." they would first solve the equation, and then determine whether the obtained solutions are integers or not. @xcite had a famous viewpoint that artificial neural networks lack this compositionality, and thus cannot serve as reliable cognitive models. current llms have achieved unprecedented success on tasks requiring complex reasoning @xcite . we wonder whether compositionality still poses a significant challenge for llms? * toward this goal, we construct a new math-trap dataset by introducing carefully designed logical traps into the original problems of the math @xcite and gsm8k @xcite datasets. for example, by modifying the original problem "find the solution of the equation x 2 + @xmath1," the model needs to combine (a) the knowledge involved in the original problem (how to solve quadratic equations with one variable) and (b) the knowledge about the trap (the definition of integers) to handle these trap problems (in fact, the original equation has no integer solutions). another reason for evaluating compositionality through trap problems is that these problems rarely appear in the real world, so it is unlikely that llms provide the correct answers solely by following the trained reasoning paths. we conduct comprehensive tests on leading llms and recruit 43 undergraduate students from top universities as human controls. we find that llms and humans exhibit strikingly different behavioral patterns when dealing with trap problems. despite possessing both (a) and (b) knowledge components, llms fail to spontaneously compose them to handle trap problems, while humans can. this suggests that tasks requiring compositional generalization natalia sold 48 clips in april and half as many clips in may. how many clips did natalia sell altogether in april and june ? conceptual problem given the sales figures for may and june, can the sales figures for april and june be calculated? ## background and definition in this section, we provide the definition of compositionality discussed in this paper, based on hilbert's formal deductive systems @xcite : definition 1. @xcite this system consists of (1) a syntax g, specifying which derivation statements are legal, (2) a set of inference rules r † , clearly stating how new facts (or theorems) can be derived from existing facts (axioms or already proved theorems), and (3) axioms: a predetermined set a of established facts. under this deductive system, reasoning is defined as the process of deriving new facts from existing facts and rules. definition 2. (compositionality in mathematical reasoning) suppose problem sets q 1 , q 2 , q 3 are described using the same syntax g, and q 1 and q 2 can derive final answers through tuple (r 1 , a 1 ), (r 2 , a 2 ) respectively, while q 3 requires reasoning using r 1 ∪ r 2 based on a 1 ∪ a 2 (or a subset) to derive the final answer. if a reasoning engine can solve q 1 and q 2 , we say it possesses compositionality if it can solve q 3 . ## the mathtrap dataset existing datasets for evaluating compositionality are limited to symbolic reasoning with semantic decoupling @xcite . however, the semantics of language play a crucial role in the reasoning process of llms @xcite . our mathtrap dataset aims to evaluate the compositionality of llms on semantically rich math word problems. more importantly, the 'unseen' feature of our dataset prevents models from simply following the trained reasoning paths to arrive at solutions. ## conclusions this paper investigates the compositional generalization of llms in mathematical reasoning. by introducing traps into the problem descriptions, we construct novel "trap problems" that cannot be solved by merely following trained reasoning paths for llms. experiments on mathtrap demonstrate that llms fail to spontaneously combine their learned knowledge to reason about trap problems. although this limitation can be mitigated through external interventions, there remains a significant gap compared to the compositional generalization capabilities of humans.