The Magic of IF : Investigating Causal Reasoning Abilities in Large Language Models of Code (2023)

Abstract: Causal reasoning, the ability to identify cause-and-effect relationship, is crucial in human thinking. Although large language models (LLMs) succeed in many NLP tasks, it is still challenging for them to conduct complex causal reasoning like abductive reasoning and counterfactual reasoning. Given the fact that programming code may express causal relations more often and explicitly with conditional statements like “if“, we want to explore whether Code-LLMs acquire better causal reasoning abilities. Our experiments show that compared to text-only LLMs, Code-LLMs with code prompts are better causal reasoners. We further intervene on the prompts from different aspects, and discover that the key point is the programming structure. Code and data are available at https://github.com/xxxiaol/magic-if .

## introduction human beings rely heavily on the capacity for causal reasoning @xcite . people understand the observed facts, predict future events, and speculate about what might have happened if things had been different with the help of their causal reasoning skills. for instance, when we go home and find a mess, we probably want to figure out why it happened. if we determine that a bird flew into the house, we might then consider whether the mess could have been avoided if we had closed the window. although large language models (llms) demonstrate great language understanding and generation abilities, it is still challenging for them to perform complex causal reasoning such as the example above. powerful llms are able to understand single cause-and-effect relations @xcite what is a plausible hypothesis given the observations? what if the counterfactual event happens? ## modeling causal structure with code we convert the input of causal reasoning tasks into the form of code prompt for code-llms to understand better. we expect the prompts to meet two requirements: 1) clearly represent the causal relationships between events, and 2) as most code-llms only support generating at the end, the target output should appear at the end of the prompts. the first requirement is addressed with conditional statements. however, for the second, the target prediction is not always the last part of the conditional statements, e.g., in abductive reasoning we want to predict the hypothesis, which is the condition in the if structure. to address this, we uniformly use functions to represent events. as shown in figure 2 , the causal structure is described in the main function. all the event functions are listed afterwards, abductive reasoning. abductive reasoning requires models to generate a plausible hypothesis h given the observations: premise p and ending e. the chronological order of these three events is @xmath0, and the hypothesis causes the ending to occur. in figure 2 , we regard the task definition as an instruction and place it as a comment at the beginning of the prompt. the causal structure is represented in the main function like: executing the premise, and if the hypothesis is met, executing the ending 2 . the content of each event is presented as a comment of its function. the hypothesis function is placed at the last, leaving for models to complete. the generation process stops with a line break. counterfactual reasoning. counterfactual reasoning aims to rewrite a story under a counterfactual condition. as in figure 1 , the input consists of four parts: the premise p , the initial context c 1 , the original ending e 1 , and the counterfactual context c 2 . models are asked to generate the counterfactual ending e 2 that minimally modifies the original ending e 1 and is coherent with the counterfactual context c 2 . the causal relationships are represented with the if-elif structure. the premise p is executed first, and then if the initial context c 1 is met, the original ending e 1 is executed; otherwise, if the counterfac2 although not entirely accurate, this approximates the actual underlying causal relationships. ## evaluation datasets. we experiment on the art dataset @xcite for the evaluation of abductive reasoning, and the timetravel dataset @xcite for counterfactual reasoning, with 3,561 and 1,871 test instances, respectively. models. we experiment with codex @xcite , trained on a blend of code and text, as the code-llm. the specific version is code-davinci-002. we compare with two llms: the latest versions of gpt-3 @xcite text-davinci-002 and text-davinci-003 (referred to as davinci 002 and davinci 003 ). both of them originate from codex and are tuned with instructions. we follow openai's default settings in codex and davinci decoding, and the text prompts for davinci are in we also compare with previous unsupervised methods on these tasks, including delorean @xcite , cold @xcite , diffu-sion @xcite , cgmh @xcite , and educat @xcite 3 . appendix a.3 bleu4 rougel cider bertscore codextext 11.7 37.5 78.5 62.5 codexcode 13.7 39.6 81.8 64.9 codex * code 16.5 42.0 91.6 66.3 davincitext 4.9 27.0 26.6 56.8 davincicode 6.7 31.1 46.2 59.9 davinci * code 9.0 35.0 64.0 62.2 (a) abductive reasoning. bleu4 rougel bertscore codextext 55.1 61.3 77.8 codexcode 66.8 70.0 82.5 codex * code 73.3 74.7 85.3 davincitext 49.0 54.7 73.0 davincicode 40.4 48.5 70.5 davinci * code 43.7 52.0 72.8 (b) counterfactual reasoning. ## what are crucial in code prompts? to paint a better picture of the key points in the code prompts, we intervene on the prompts from four aspects and measure the influences of the interventions. the four aspects we select are information, structure, format, and language. the former two, the prior information provided and the programming structure of functions, are contentrelated; the latter two, the code format and programming languages, are form-related. an ideal model should rely on the content and be insensitive to form perturbations. the interventions are described below, with examples in figure a .2. information. we study two types of prior information: task instructions and function names. in no instruction, we remove the task instruction from the prompts. in function name perturbation, we replace original function names with anonymous functionx. for example, we replace premise() and hypothesis() in figure 2 with functiona() and functionb(), respectively. it eliminates the information in function names and only allows models to learn the event relations from programming structures. structure. the first way to intervene in the programming structure is to convert the conditional structures into sequential structures, referred to as sequential structure. the events are executed sequentially, like premise(), hypothesis(), ending() in abductive reasoning. in the second way called disruption, we randomly disrupt the positions of the functions in the conditional structure. for instance, if hypothesis(): ending() can be disrupted into if ending(): hypothesis(). we also apply the function name perturbation in disruption to eliminate the impact of function names. format. we test three formats besides the original one: class, print and return. the first one converts the original code into a class. we define the programming structure in the __init__ method, and move the event functions into the class. in print, we represent the content of events as a string and print it in the function body, like def premise(): print("the smiths ..."). and in return, the string is the return value of event functions. language. we also convert the original python programs into two other languages, java and c, to evaluate the influence of programming languages. intervention results. we evaluate the influence of interventions on abductive reasoning in codex is quite robust towards format and language changes. settings like class and java are even better than the original one, revealing that the performance can be further improved with delicate prompt engineering. ## conclusion we investigate the causal reasoning ability of code-llms. with code prompts of conditional statements, code-llms achieve great performance in abductive and counterfactual reasoning, outperforming text-only llms significantly. our study on different aspects of code prompts shows that providing a reasonable causal structure in code can help generate plausible outputs, and code-llms are robust towards format perturbations.