Diverse Multi-tool Aggregation with Large Language Models for Enhanced Math Reasoning (2025)

Abstract: Tool usage is a proven technique for developing high-performance reasoning in large language models (LLMs). Our work is focused on emphasizing the utility of leveraging multiple diverse tools for complex reasoning tasks. We present \textbf{Multi-TAG} , a \textbf{Multi} - \textbf{T} ool \textbf{AG} gregation-based LLM framework that utilizes multiple diverse tools to solve complex math problems over multiple reasoning steps. At each reasoning step, \textbf{Multi-TAG} invokes multiple tools and accepts the solution of the respective step by tools that have majority agreement on the final answer estimate. \textbf{Multi-TAG} strongly outperforms several standard baselines that use individual tools with the same number of runs, highlighting the importance of multi-tool invocation for solving complex reasoning tasks. We also show that naive aggregation of multiple tools at each reasoning step also leads to substantial improvements of up to 35% accuracy. \textbf{Multi-TAG} then further improves these gains by 7.4% on average on MATH500, AIME, AMC, and OlympiadBench.

## introduction large language models (llms) have demonstrated remarkable capabilities across a wide variety of tasks, with reasoning emerging as a core area of research @xcite @xcite . in particular, imbuing llms with the ability to perform complex mathematical reasoning remains an active challenge 1 multi-tag github will be open-sourced soon. @xcite . to address this challenge, tool-augmented llm (talm) frameworks such as program-aided language models (pal) @xcite , program-of-thought (pot) @xcite , tool-integrated reasoning agent (tora) @xcite , and mathsensei @xcite equip llms with external tools such as python code execution or wolframalpha querying. while previous talm frameworks have shown notable progress on simpler math benchmarks such as gsm8k @xcite , their performance plateaus and remains low on more complex benchmarks such as math500 @xcite , aime, amc, and olympiad-bench @xcite . inference-time compute scaling approaches such as openai o1 @xcite , @xcite and rstar-math @xcite allocate more computational resources to llms at inference time to allow them to reason more methodically about problems, and also demonstrate promise towards addressing the challenge of llm math reasoning. although these approaches achieve impressive performance on complex math benchmarks, they generally require extensive finetuning, which can be especially brittle and requires training tricks and carefully tuned hyperparameters to be effective @xcite . moreover, finetuning can be prohibitively expensive, especially for larger llms, and may not be possible for proprietary llms whose apis do not support the finetuning recipes used in these approaches. furthermore, recent work has shown that these approaches exhibit non-robust reasoning behaviors in various scenarios, such as number transformations @xcite , adversarial triggers @xcite , unanswerable tasks @xcite , and executing algorithmic solutions @xcite . in this work, we propose multi-tag, a multi-tool aggregation framework that aims to address the limitations of existing talm frameworks on complex math reasoning tasks by adopting the inference-time scaling paradigm. as opposed to previous talm frameworks which select a single tool to invoke at each reasoning step, multi-tag scales up inference-time compute usage by invoking multiple tools at each reasoning step and aggregating their outputs, utilizing the consensus between different tool invocations to ensure that accurate reasoning steps are made. the core benefit of multi-tool aggregation is cross-validation of different tools' outputs. since different tools have different strengths and failure modes (see section 5.1.2), their agreement on a result provides strong evidence of its correctness. for example, a natural language reasoning tool and a python code execution tool both proposing a consistent result lends significant credence to the validity of the result, as it is improbable that both tools made unique mistakes thematic to their own weaknesses (e.g. calculation mistakes for the natural language tool and logical mistakes for the python tool) yet coincidentally arrived at the same incorrect result. by invoking and aggregating a diverse set of tools at each reasoning step, multi-tag harnesses this principle to self-validate candidates for each step, significantly improving the overall reasoning performance. beyond superior reasoning performance, another key strength of multi-tag is that it is a purely inference-time approach, making it readily applicable to any general instruction-tuned llm. in contrast, finetuning-based talm approaches such as tora @xcite or mathcoder @xcite incur a significant compute overhead for finetuning when applied to new backbone llms and cannot be applied to proprietary llms lacking finetuning apis. to demonstrate multi-tag's transferability to different llm backbones, we replicate our main results on three llms, including both open-weight and proprietary models, and observe consistent and significant performance improvements over baselines for all three llms. furthermore, multi-tag's inference compute costs are adjustable, allowing users to tune multi-tag to fit their cost-performance tradeoff requirements, as opposed to prior talm approaches where compute allocation is not easily tunable. we study how performance and compute usage vary for different multi-tag hyperparameter settings in section 5.2 and propose a simple hyperparameter heuristic to maximize performance under a fixed compute budget. our key contributions are: 1. we introduce multi-tag, a tool-augmented llm framework that effectively solves complex math reasoning tasks by aggregating multiple tool invocations at each reasoning step. the code repository for multi-tag code will be open-sourced in the near future. 2. we present extensive evaluations of multi-tag, seven simple baselines, and five state-ofthe-art talm baselines on three llm backbones and four challenging math reasoning benchmarks: math500, aime, amc, and olympiadbench. across the three llms, stateof-the-art talm baselines perform poorly, with the strongest talm baseline for each model achieving 1.3% to 6.2% lower accuracy compared to the strongest simple baseline for each model. meanwhile, multi-tag performs much better, achieving 6.0% to 7.5% higher accuracy compared to the strongest simple baseline for each model and 7.9% to 13.7% higher accuracy compared to the strongest talm baseline for each model. 3. we present comprehensive analyses highlighting the strengths of multi-tag and benchmarking the cost-performance trade-off. in particular, we analyze multi-tag's performance by problem difficulty and problem subject area in section 5.1. we also study how multi-tag's performance and compute cost vary with different hyperparameter settings and present insights for how users can tune multi-tag to extract maximal performance under various computational budgets in section 5.2. ## related work tool-augmented language models recent advancements in developing tool-augmented language models (talms) have shown promise in improving performance on reasoning tasks. frameworks such as toolformer @xcite , olagpt @xcite , and art @xcite demonstrate that allowing llms to access external tools can significantly boost performance in various domains. many previous works have also focused on developing talms for math reasoning tasks. program-aided language models (pal) @xcite and program of thoughts (pot) @xcite propose to prompt llms to generate python code solutions to math prob-lems instead of natural language chain-of-thought (cot) solutions and achieve strong performance on simple math word problem datasets such as gsm8k @xcite and svamp @xcite . mammoth @xcite finetunes models to produce both cot solutions and python code solutions to math problems. more recent works have recognized the importance of multi-step tool use for solving more challenging math problems which are too complex to solve with a single tool call. mumath-code (yin et al., 2024) and tora (gou et al., 2024) finetune models to generate solutions with multiple interleaved cot and python code reasoning traces. mathsensei (das et al., 2024) prompts llms to solve math problems by following handcrafted workflows of sequences of tool invocations. code-based self-verification prompting (zhou et al., 2023) leverages gpt4 code interpreter's built-in code execution capabilities to verify each cot reasoning step with a python script. while previous talm approaches have been effective on simpler math reasoning tasks, their use of only one tool invocation per reasoning step may limit their scalability to more complex math problems. to address this limitation, we hypothesize that allowing talms to invoke multiple tools at each reasoning step improves their capacity for solving complex math problems by enabling cross-verification and leveraging the complementary strengths of different tools. to this end, the proposed multi-tag framework aggregates outputs from multiple tools at each step, allowing different tools to validate and reinforce each part of the reasoning process, thus enhancing the accuracy of intermediate reasoning and the final solution. furthermore, talm approaches that rely on finetuning are not easily generalizable to all models, requiring significant computational resources to finetune larger models @xcite , and impossible to apply to proprietary models lacking finetuning apis. in contrast, our framework relies solely on prompting and the inherent instruction following capabilities of llms, allowing it to be seamlessly applied to any llm. inference-time compute scaling inferencetime compute scaling approaches aim to improve llm reasoning performance by utilizing more compute at inference time. self-consistency @xcite proposes a simple way to implement inference-time scaling, where multiple solutions are sampled from an llm and the most common answer is chosen. building on top of these repeated sampling approaches, other works have further proposed to train verifiers to better select the best solution from the sampled solutions. some works propose to train verifiers to judge entire solutions @xcite , whereas others propose to train verifiers to judge individual steps of solutions @xcite . inline with step-wise verification approaches, our multi-tag approach also performs verification at each reasoning step, but utilizes cross-verification between different tools' outputs instead of trained verifiers. inference scaling via long cot reasoning has also been shown to be an effective technique @xcite , although solving reasoning tasks with only cot reasoning has limitations in various settings @xcite @xcite @xcite . in a complementary direction, our work explores inference-time scaling through multi-tool aggregation to enhance the performance of talms. figure 1 provides an overview of the multi-tag system, and algorithm 1 provides an explicit pseudocode implementation. given a problem p and a figure 1 : visualization of the multi-tag framework with a consistency threshold value of 1. in the first step, after the first four executors are invoked, candidates cot (1), python (2), wolframalpha (3), and cot (4) are produced. candidates (1), (3), ( 4 ) have final answer estimate 1, while executor (2) has final answer estimate 3. the frequency of the most frequent final answer estimate, 1, is 3, while the frequency of the second most frequent final answer estimate, 3, is 1, so the consistency gap is 3 -1 = 2, which is greater than the consistency threshold value. hence, executor invocation terminates. to select a candidate, first the candidates (1), ( 3 ), (4) are shortlisted as they reach the most frequent final answer estimate of 1. then, (4) is selected as it has the shortest solution completion. in the second step, only two executors were invoked for the consistency gap to exceed the consistency threshold value. candidate (2) was selected due to having the shorter solution completion. this process repeats until the selected step reaches a final answer for the problem. the full generated solution to the problem is the concatenation of all the selected steps. after executor invocation is completed, each candidate s i p is appended to the current partial solution, forming a candidate partial solution cand @xmath0, s 2 , . . . , s p-1 , s i finally, to select the best cand i to serve as the next step in the current partial solution, a two-step selection procedure is employed. in the first step, the most frequent final answer estimate @xmath1, est 2 , . . . , est m×t }) is identified, and all candidates cand i such that est @xmath2. similar to self consistency's motivation, the more candidates that reach consistent final answers, the more confident we can be about the candidates' accuracy. in the second step, the shortlisted candidate with the shortest solution completion (measured in number of llm tokens) is selected to be the next step in the current partial solution. intuitively, selecting this step would lead to the most concise solution, improving multi-tag's compute efficiency. furthermore, we find in our ablation analyses in appendix b.3 that using this second selection step also improves performance. this finding can be intuitively explained by the occam's razor principle, that concise explanations should be favored over complex ones, and is consistent with recent work demonstrating that concise reasoning can sometimes lead to stronger performance @xcite . we perform ablation tests on our two-step selection procedure in appendix b.3 and find empirically that both steps are necessary to achieve maximum performance. furthermore, we demonstrate that the second step additionally improves inference efficiency, and removing it results in substantially higher inference costs. ## conclusion in this paper, we present multi-tag , a novel toolaugmented llm framework for math reasoning. unlike previous talm frameworks, multi-tag scales up the inference-time compute allocated by allowing the llm to invoke and aggregate multiple tools at each reasoning step. as a result, multi-tag achieves superior results on four complex math reasoning benchmarks, outperforming the strongest baselines by 6.0% to 7.5% over three different backbone llms. furthermore, multi-tag is widely applicable, enabling the use of any