A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models (2023)

Abstract: We have recently witnessed a number of impressive results on hard mathematical reasoning problems with language models. At the same time, the robustness of these models has also been called into question; recent works have shown that models can rely on shallow patterns in the problem description when generating a solution. Building on the idea of behavioral testing, we propose a novel framework, which pins down the causal effect of various factors in the input, e.g., the surface form of the problem text, the operands, and math operators on the output solution. By grounding the behavioral analysis in a causal graph describing an intuitive reasoning process, we study the behavior of language models in terms of robustness and sensitivity to direct interventions in the input space. We apply our framework on a test bed of math word problems. Our analysis shows that robustness does not appear to continuously improve as a function of size, but the GPT-3 Davinci models (175B) achieve a dramatic improvement in both robustness and sensitivity compared to all other GPT variants.

## introduction many natural language understanding situations, such as understanding the financial news, require reasoning with text that includes numbers. however, such mathematical reasoning is challenging for nlp models @xcite . mathematical reasoning for text has been an active area of research for a while @xcite @xcite , and has also emerged as a key task to track the capabilities of large language models (llms) in recent years @xcite @xcite . however, despite the impressive performance of llms on various math reasoning benchmarks (e.g., after do-intervention @xmath0.085 @xmath1.001 ## problem setup we consider a dataset d of math word problems (mwps), where each mwp is denoted as a question q. q is a list (t , n ) consisting of a question template t and an ordered list of operands @xmath2, n 2 , . . . , n m ). each question template t := (o, s) further contains two types of information: a set of arithmetic operations o implicitly expressed in the question, and the text surface form s irrelevant to the arithmetic operations. o incorporates the information relative to the operations as a collection of tuples {(o 1 , i 1 , j 1 ), (o 2 , i 2 , j 2 ), . . . }, where o @xmath3, -, ×, ÷} (@xmath4, j @xmath5. 2 the ground-truth result @xmath6, which represents the application of all the operators in o to the respective operands. we illustrate the factors in q and their inter-dependency in the causal graph in figure 2 . a two-operand instance q of q in this form from @xcite is: template t: mark has n 1 trees in his backyard. if he plants n 2 more, how many trees will he have? our goal is to quantify the robustness of a model m on the set of problems @xmath7. ideally, d should be a dataset not seen by the model during training. we assume that a model takes q as input and predicts a probability distribution of the result r: p(r | t, n). our formulation below will be easier to understand using this finite discrete set and can be generalized to any kind of data pairing a natural language template with a function that maps a set of operands to a result (e.g., a python program; @xcite . ## a causal framework in this section, we describe our framework in three steps. first, we define the idea of model robustness on mwps. then, we identify possible dointerventions @xcite that we can perform. finally, we describe the causal effects that we measure to quantify the robustness of various models. ## experimental setup in this section, we describe the data used to perform the interventions and to measure the causal effects. ## results our analyses focus primarily on two-operand problems (sections 5.1 and 5.2) and later extend to more complex problems that involve three operands (section 5.5) for the models that perform best on the two-operand test bed. we compare the direct causal effect dce and the total causal effect tce of n and t on r. dce represents the undesired effect for a model to being mistakenly responsive to a change in n or t not leading to a change in the result g (low robustness), whereas higher values of tce indicate a higher ability of the model to correctly adjust the probability weight assigned to the new solution g ′ after the intervention (high sensitivity). ## conclusion we developed a framework to disentangle and separately measure the effect of different factors influencing the predictions of llms for math reasoning. our results indicate that a drastic increase in both robustness and sensitivity emerges in the gpt-3 davinci models. additionally, we study the contribution of size and instruction tuning in the models of the llama family, observing that the alpaca instruction tuning, while increasing the model's robustness, does not significantly improve the overall performance. our framework provides a formalized theory of behavioral testing for math reasoning models and opens new future directions to design behavioral tests of models in a principled way.