Code Prompting Elicits Conditional Reasoning Abilities in T ext+ C ode LLM s (2024)

Abstract: Reasoning is a fundamental component of language understanding. Recent prompting techniques, such as chain of thought, have consistently improved LLMs’ performance on various reasoning tasks. Nevertheless, there is still little understanding of what triggers reasoning abilities in LLMs in the inference stage. In this paper, we investigate the effect of the input representation on the reasoning abilities of LLMs. We hypothesize that representing natural language tasks as code can enhance specific reasoning abilities such as entity tracking or logical reasoning. To study this, we propose code prompting, a methodology we operationalize as a chain of prompts that transforms a natural language problem into code and directly prompts the LLM using the generated code without resorting to external code execution. We find that code prompting exhibits a high-performance boost for multiple LLMs (up to 22.52 percentage points on GPT 3.5, 7.75 on Mixtral, and 16.78 on Mistral) across multiple conditional reasoning datasets. We then conduct comprehensive experiments to understand how the code representation triggers reasoning abilities and which capabilities are elicited in the underlying models. Our analysis on GPT 3.5 reveals that the code formatting of the input problem is essential for performance improvement. Furthermore, the code representation improves sample efficiency of in-context learning and facilitates state tracking of entities.

## introduction reasoning is a fundamental component of both human and artificial intelligence (ai) and the backbone of many nlp tasks. recently, intensive studies have been performed on different aspects or types of reasoning such as mathematical reasoning @xcite @xcite , various kinds of logical reasoning figure 1 : code prompting converts a natural language problem into a code prompt and prompts a large language model with such code to generate an answer. @xcite @xcite , and commonsense-focused reasoning @xcite @xcite . conditional reasoning, a primary yet complex reasoning ability that draws alternative conclusions depending on the fulfillment of certain conditions, remains understudied. these conditions are stated in the text, making the problem self-contained, which allows us to study the semantic inferencing capabilities of the underlying model, i.e., identifying relevant premises and ascertaining the presence of total evidence @xcite without the requirement for, and confounding effects of external knowledge. conditional reasoning is also a fundamental form of reasoning useful in many practical scenarios, such as answering real-world questions about the eligibility for a visa or a loan. despite the recent introduction of some benchmarks @xcite @xcite , conditional reasoning abilities of llms remain understudied. recently, researchers have analyzed the synergies between llms and symbolic interpreters to improve performance on reasoning tasks @xcite @xcite . these works transform structured reasoning problems, such as mathematic or symbolic reasoning, into code and run it on an external interpreter. in such a setup, llms are mainly focused on natural language representation aspects and planning how to solve the problem, while the actual logical reasoning is offloaded to an external execution module, confounding our understanding of the reasoning in particular, the fundamental questions of what contributes to the reasoning abilities and how reasoning abilities are triggered in llms remain open. nevertheless, pretraining on code is considered an important component that contributes to and explains the improved reasoning ability of llms. state-of-the-art llms such as gpt 3.5 @xcite , gpt 4 (openai, 2023), mixtral @xcite , and mistral 7b @xcite have been pretrained on both text and code and have demonstrated considerable boosts in many reasoning benchmarks. in this work, we analyze whether one can elicit improved conditional reasoning abilities in llms by merely changing the input format, i.e., from text to code. we constrain our experiments to text+code llms to run text and code inputs on the same underlying model. in this way, we can avoid the confounding factor of different pretraining data of specialized text and code llms. to understand the benefit of code as an intermediate representation, we devise a chain of prompts, code prompting, that transforms a natural language (nl) task into code and directly prompts the llm with the generated code. the code contains the logical structure needed to solve the problem, along with the original natural language text as code comments. an illustration is provided in figure 1 . our contributions are summarized as follows: augmenting text with code. most works that generate code to solve natural language tasks use an external symbolic interpreter to run the resulting code. @xcite and @xcite showed consistent gains on mathematical problems, symbolic reasoning, and algorithmic problems by using llms aided by external code interpreters. @xcite further report improvements in boolean multi-hop qa, planning, and relational inference. in contrast, @xcite used an external automated theorem prover with declarative code and showed consistent gains w.r.t. imperative code-interpreter-aided llms on arithmetic reasoning, logical reasoning, symbolic reasoning, and regex synthesis tasks. @xcite did not use any interpreter and instead created programs composed of multiple subroutines and used smaller specialized models to run them. in this way, they outperform text prompts on text llms for factchecking tasks. lastly, @xcite runs pieces of code in an llm to update the program state when the python interpreter fails due to a code exception and shows performance gains on big-bench hard @xcite . all these works investigate how to best use an external symbolic interpreter to aid an llm in solving reasoning tasks, i.e., they run code and therefore have a program state with variables and its values. however, we do not employ any external symbolic reasoner, and we do not run code. we investigate the reasoning abilities of llms under different input representations (i.e., text and code). our code prompts are not executed; they are simply read by the llm and used to generate a natural language answer. some works suggest that code llms may possess superior reasoning abilities than text llms. @xcite investigate whether code llms are superior at structured reasoning than text llms. they observe that code llms can generate graphs that link commonsense concepts better than text llms. @xcite investigate code prompts in abductive and counterfactual reasoning tasks and report superior results than text prompts on code-davinci @xcite , a code llm. however, code prompts exhibit mixed results on text-davinci-002 @xcite , a text llm. we attribute this to the fact that while this model includes some code in its pretraining corpus, it is not explicitly trained for code generation and, in general, performs poorly on code generation tasks @xcite . therefore, the effect of the input representation on the reasoning abilities of text+code llms remains unclear. furthermore, the reasons behind the superior performance of code prompts in code llms also remain unclear. in our work, we aim to answer whether code prompts can elicit conditional reasoning abilities in text+code llms and the reasons behind this. to the best of our knowledge, only the work of @xcite investigates the conditional reasoning abilities of llms. however, they only analyze the abilities of text llms after training them on conditionalqa @xcite . ## code prompting we posit that each llm encodes a set of capabilities, such as mathematical, logical, or conditional reasoning. however, not all of them are used for every input instance, even if they would be useful. we hypothesize that the input representation plays a pivotal role in eliciting such capabilities. prior works show that llms trained on a combination of text and code exhibit superior reasoning abilities @xcite @xcite . therefore, we conjecture that a code representation of a natural language (nl) problem may trigger some of these reasoning abilities encoded in text+code llms. more formally, we wonder whether exists some space s foot_0 with an associated function f that transforms a natural language problem @xmath0, such that, when prompting an llm with the representation of p in such space yields better results according to some evaluation function σ. we fix s to the programming language space and define code prompts f (p) as prompts that model a natural language problem with code. we also define f as a prompt that transforms the nl text into code. f (p) code follows the original nl text as much as possible. we use a simple structured code that contains the logical structure needed to solve the problem, along with the original nl text as code comments. in particular, it creates variables for key entities in the question and documents and if blocks for each conditional statement in the documents. figure 2 exemplifies this transformation and appendix d provides more details of the code features. lastly, we define code prompting as llm (f (p)), a chain of prompts that i) transform the nl text into code, and ii) use this code to prompt the llm to generate the answer in natural language. figure 1 illustrates this pipeline. it is important to note that the code is not executed per se and therefore, there is no program state. we simply prompt the llm with the code and ask the llm to generate a natural language answer based on the content of such code. this setup allows us to investigate the effect of the input representation on text+code llms. ## experiments we devise a set of experiments to analyze and quantify whether the code representation of a natural language prompt (i.e., code prompts) elicits conditional reasoning abilities and why. we first compare the performance of the two prompting methods -text prompts and code prompts on three llms across three datasets ( §5.1). we then conduct extensive ablation experiments on the dev set of the datasets with gpt 3.5, the best-performing and largest model, to understand the reason behind the performance gain from code prompting. in particular, we study whether code syntax or the implicit text simplification from the code translation is what improves performance (section 5.2). we also check if the improvement is caused by the models merely being exposed to code within prompts and not necessarily the instances translated to code (section 5.3). furthermore, we show that code prompting is more sample efficient (section 5.4) when compared to text prompting and that models prompted with code exhibit superior state tracking capabilities (section 5.5). lastly, we conduct a human evaluation that confirms the faithfulness of the generated code in section 6. ## human analysis of the generated code we conduct a small human evaluation to confirm the faithfulness of the generated code to the source natural language text. we evaluate the code generations of all our models on ten random samples from the dev set of condqa, sharc, and bgqa (in particular, we use bgqa-1 partition). we check for perfect translations, and for the failing cases, we analyze the errors. ## conclusions and future work this work demonstrates that the code representation of a natural language task (i.e., code prompts) can elicit reasoning abilities in large language models (llms) of text and code. these code prompts contain the original natural language (nl) formulation as a code comment and code that formulates the logic of the text. to create these code prompts, we use in-context learning to teach an llm how to conduct such a transformation automatically. through multiple experiments on three llms and three datasets, we show that code prompts trigger conditional reasoning abilities, with large performance gains w.r.t. text prompts (up to 22.52 percentage points on gpt 3.5, 7.75 on mixtral, and 16.78 mistral). our experiments reveal that even simple code can be beneficial as long as it closely follows the semantics of the nl text and is accompanied by the original nl text. we also show that code prompts are more sample-efficient than text prompts and that their performance boost stems from their superior ability to identify and track the state of key variables or entities, a central aspect of the logical dimension of semantic inference. in our future work, we plan to extend to a wider range of reasoning abilities, such as multi-hop reasoning, to understand the capacity and generalizability of code prompting. we also plan to investigate how pretraining on text, code, and text+code affects the triggering of llms' reasoning abilities.