From Good to Great: Improving Math Reasoning with Tool-Augmented Interleaf Prompting (2024)

Abstract: This paper investigates the performance of Large Language Models (LLMs) and Tool-augmented LLMs in tackling complex mathematical reasoning tasks. We introduce IMR-TIP: Improving Math Reasoning with Tool-augmented Interleaf Prompting, a framework that combines the strengths of both LLMs and Tool-augmented LLMs. IMR-TIP follows the “From Good to Great” concept, collecting multiple potential solutions from both LLMs and their Tool-Augmented counterparts for the same math problem, and then selecting or re-generating the most accurate answer after cross-checking these solutions via tool-augmented interleaf prompting. The framework incorporates two key aspects: self-prompt and tool-augmented interleaf prompting (TIP). The former allows LLMs to autonomously refine and improve an initial prompt related to tool usage, while the latter enables LLMs to derive the final answer by dynamically analyzing the problem, cross-checking potential solutions, and revising previous reasoning hints in an interleaved manner. Experimental analysis shows that IMR-TIP achieves enhanced mathematical capabilities and outperforms traditional LLMs and tool-augmented LLMs in accuracy and reasoning diversity on math reasoning tasks. For instance, IMR-TIP can improve Tool-augmented ChatGPT on GSM8K-Hard from 56.0% to 65.2 %.

## introduction large language models (llms) @xcite @xcite @xcite such as gpt4 have exhibited remarkable performances across a wide array of downstream tasks. they can effortlessly tackle downstream tasks by conditioning on a scant number of in-context exemplars or plain natural language task descriptions @xcite . notwithstanding these significant advancements, even the most extensive llms are although these approaches have led to signif-icant improvements overall, a pertinent question arises: can these tool-augmented llms outperform their traditional llm-cot counterparts consistently? to explore this question, we conduct a pilot analysis at first: we test both llms-cot and its tool-augmented counterpart (calculatoraugmented) on gsm8k dataset, separately. based on the aforementioned analysis, we aim to capitalize on the strengths of both llms and tool-augmented llms to further enhance their mathematical capabilities. in this work, we introduce imr-tip: improving math reasoning with tool-augmented interleaf prompting. imr-tip is a framework that follows the idea "from good to great" that first collects multiple potential solutions from llms and their tool-augmented approaches given the same math problem, and then selects the most correct answer or re-generates new answer after cross-checking these solutions in a tool-augmented interleaved manner. to achieve this, we design imr-tip in two aspects: @xcite we first propose self-prompt to address the challenge of crafting clear, diverse and effective tool-based prompts. it allows llms to autonomously refine and improve an initial prompt related to tool usage, resulting in enhanced prompts. moreover, the self-prompt can yield multiple diverse toolbased prompts through iterative runs without requiring repetitive manual modifications, eliciting varied reasoning paths (solutions) in llms. (2) then given multiple solutions, we introduce toolaugmented interleaf prompting (tip), a paradigm combining reasoning, acting, and external tools in an interleaved manner to arrive at the correct answer. tip allows llms to dynamically analyze the problem, cross-check potential solutions, and re-evaluate even revising previous reasoning hints through action and thought. our technical contributions are summarized as follows: ## methodogy in this section, we aim to illustrate our method in detail, as seen in figure 1 and figure 2 . we first review the problem formulation of math problem reasoning. then we introduce our proposed selfprompt and tool-augmented interleaf prompting methods, sequentially. ## experiments in this section, we first review our testing datasets and evaluation metrics. then we briefly introduce our experimental settings that include backbones, and our prompts. at last, we will present our main results and ablation studies. ## analysis imr-tip vs. self-consistency given multiple solutions from llms and tool-augmented llms, two intuitive ways to derive the answer are: 1) self-consistency; 2) direct select. as aforementioned, we have three versions of self-consistency: cot-sc, tool-sc and mix-sc. figure 3 shows the comparison between ours and these approaches on three datasets. we can observe that tool-sc and mix-sc perform comparably but they perform much better than cot-sc, just as tool-augmented llms surpass llm-cot. across two distinct backbone models, imr-tip consistently outperforms other methods on each dataset. ## conclusion this work introduces imr-tip, a framework designed to capitalize on the strengths of both conventional and tool-augmented llms, enhancing their mathematical capabilities. imr-tip employs self-prompt to refine its initial prompts. additionally, it utilizes tool-augmented interleaf prompting (tip) to reach the correct answer. this process involves interleaved action and thought, which encompasses analyzing the problem, observing, crossverifying different solutions, and even rethinking previous answer hints. through rigorous experimentation and comprehensive analysis of math reasoning tasks, imr-tip demonstrates a substantial enhancement over established baselines.