Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning

## Abstract

Abstract Existing methods usually leverage a fixed strategy, such as natural language reasoning, code-augmented reasoning, tool-integrated reasoning, or ensemble-based reasoning, to guide Large Language Models (LLMs) to perform mathematical reasoning. Our analysis reveals that the single strategy cannot adapt to problem-specific requirements and thus overlooks the trade-off between effectiveness and efficiency. To address these issues, we propose Planning and Routing through Instance-Specific Modeling (PRISM), a novel framework that decouples mathematical reasoning into two stages: strategy planning and targeted execution. Specifically, we first curate a multi-strategy preference dataset, which we call MathStrat , capturing correctness, process quality, and computational efficiency for each problem–strategy pair. Then, we train a lightweight Strategy Adapter based on the dataset to obtain confidence distributions over the mentioned four reasoning strategies. At inference time, an adaptive routing policy dynamically tailors the reasoning approach based on predictor confidence. It directs the model to use single-strategy execution for high-confidence predictions, dual-strategy verification for competitive scenarios, or comprehensive multi-strategy exploration for uncertain cases. Extensive experiments across five mathematical reasoning benchmarks demonstrate that PRISM consistently outperforms individual strategies and ensemble baselines, achieving improvements ranging from 0.9% to 7.6% across different base models. The adaptive routing approach shows particularly strong benefits for mathematical reasoning tasks across diverse model architectures. Our code is released at https://github.com/reml-group/PRISM .

## 1 Introduction

Large Language Models (LLMs), such as ChatGPT and Qwen, have achieved significant advancements across diverse natural language processing tasks (Ma et al., 2025a; b). Notably, they have demonstrated strong performance in mathematical reasoning—a long-standing and challenging domain that requires precise logical inference, symbolic manipulation, and multi-step problem-solving (Gou et al., 2024b). Existing approaches to improving mathematical reasoning in LLMs can be broadly divided into three categories: (1) designing effective prompting methods for frozen LLMs (Trivedi et al., 2025), (2) developing strategies to enhance the capability of frozen LLMs (Didolkar et al., 2024), and (3) post-training LLMs on domain-specific data (Xia et al., 2025). Among these, the method of enhancing frozen LLMs through inference-time mechanisms such as chain-of-thought refinement (Wei et al., 2022b) and tool invocation (Xie et al., 2025) has garnered considerable attention due to its ease of deployment. Although these methods have achieved significant success, they still face two challenges.

Challenge 1: One strategy does not fit all. Existing methods primarily rely on isolated reasoning strategies, including Natural Language Reasoning (NLR) (Wang et al., 2024), Code-Augmented Reasoning (CAR) (Ye et al., 2024), Tool-Integrated Reasoning (TIR) (Li et al., 2024), and Ensemble-Based Reasoning (EBR) (Ranaldi et al., 2024), to enhance the mathematical reasoning of frozen LLMs. As illustrated in Figure [1](https://arxiv.org/html/2509.24377v1#S1.F1), we evaluate the performance of these individual strategies in boosting the reasoning ability of Qwen-2.5-math across four question types sampled from MATH (Hendrycks et al., 2021), GSM8K (Cobbe et al., 2021), and SVAMP (Patel et al., 2021). Our analysis reveals that no single strategy consistently outperforms others across diverse problem categories. This performance variance underscores a key limitation: rigidly adhering to a fixed reasoning paradigm fails to fully unlock the latent capabilities of frozen LLMs and hampers the adaptability to various problem types.

Challenge 2: Trade-offs between efficiency and effectiveness are overlooked. Current approaches (Xin et al., 2025; Zhang & Xiong, 2025) often disregard the computational cost, latency, and resource efficiency of reasoning strategies. As shown in Figure [1](https://arxiv.org/html/2509.24377v1#S1.F1), we report the normalized inference efficiency of different strategies for enhancing the reasoning ability of Qwen-2 across four question types. Notably, no single strategy consistently achieves the best efficiency. This observation suggests that a fixed reasoning paradigm leads to suboptimal deployments, where substantial computational expense does not yield commensurate improvements in accuracy.

Figure: Figure 1: Performance of four reasoning strategies on four problem slices. Each slice (e.g., Number Theory, Geometry) comprises over 100 instances drawn from the MATH, GSM8K, and SVAMP benchmarks.
Refer to caption: image/1.8.png

To address these challenges, we introduce Planning and Routing through Instance-Specific Modeling (PRISM), a framework that decouples mathematical reasoning into two core stages: strategy planning and targeted execution. Specifically, we propose a data construction approach based on multi-strategy performance profiling, which systematically evaluates diverse reasoning strategies on each problem instance to generate fine-grained suitability distributions rather than single-strategy labels. To achieve this, we execute four distinct reasoning strategies (i.e., NLR, CAR, TIR, and EBR) on problems from standard benchmarks like MATH and GSM8k. Each resulting solution trajectory is then evaluated using a multi-faceted scoring function that considers correctness, process quality, and efficiency to generate per-strategy suitability scores. These raw scores are then transformed into a soft target distribution via a temperature-scaled softmax function. We then train a lightweight Strategy Adapter by minimizing the Kullback-Leibler (KL) divergence between its output and this target distribution, which encourages the model to capture the relative suitability of strategies for each instance. At inference time, the output from the predictor drives our problem-aware strategy routing that reconciles the efficiency-effectiveness trade-off through confidence-based execution paths: high-confidence predictions trigger streamlined single-path execution; competitive scores between strategies invoke dual-path verification for robustness; and diffuse uncertainty defaults to comprehensive multi-path exploration. Through this confidence-guided orchestration, PRISM achieves both strategic flexibility and computational efficiency, selecting the most suitable reasoning approach for each problem while scaling computational effort according to prediction certainty.

To verify the effectiveness and superiority, we evaluate PRISM across five standard mathematical reasoning benchmarks, including MATH500, GSM8K, AQUA-RAT (Ling et al., 2017), SVAMP, and ASDiv (Miao et al., 2020). Our experiments show that PRISM consistently delivers significant performance gains. Notably, on the challenging MATH benchmark, our method achieves an accuracy of 53.2%, surpassing the best-performing single-strategy baseline (TIR) by 3.1% absolute. Furthermore, it outperforms the standard ensemble-based approach (EBR) by a margin of 2.5% absolute, demonstrating the superiority of pre-execution routing over post-hoc aggregation.

Our main contributions are as follows:

- 1.
We introduce PRISM, a novel framework that decouples the mathematical reasoning process into two distinct stages: strategy planning and targeted execution. This is operationalized through a lightweight meta-predictor trained on MathStrat, our curated dataset of $\sim$13,000 instances that provides rich, multi-faceted supervision signals that capture the relative suitability of various reasoning strategies.
- 2.
We design a dynamic, verifier-free routing policy for inference. This policy interprets the predictor’s output to adaptively select among single, dual, or multi-path execution modes, providing a principled mechanism to balance performance with computational cost.
- 3.
We conduct extensive experiments across five standard mathematical benchmarks. The results demonstrate that PRISM consistently and significantly outperforms all single-strategy baselines and a standard ensemble method, validating the superiority of our problem-aware routing approach.

## 2 Related Work

Natural Language Reasoning The dominant paradigm for complex reasoning in LLMs is Chain-of-Thought (CoT), which externalizes intermediate steps in natural language (Wei et al., 2022a). Recent efforts to improve NLR have focused on enhancing the quality of process data used for fine-tuning, with representative approaches such as bootstrapping via question back-translation (Yu et al., 2024; Lu et al., 2024b) and evolutionary rewriting of instructions (Luo et al., 2025). While effective for symbolic deduction, NLR’s reliance on unstructured text makes it prone to arithmetic and logical errors in computationally intensive problems.

Code-Augmented Reasoning To address the computational limitations of NLR, CAR reframes mathematical problems as program generation tasks, offloading calculations to a deterministic code interpreter (Zhang et al., 2024). This is often implemented through prompting paradigms like Program-of-Thoughts (PoT) (Chen et al., 2023) or by fine-tuning models on interleaved text and code, as in Program-Aided Language Models (PAL) (Gao et al., 2023b). CAR excels at numerical precision but remains dependent on the initial natural language understanding for problem decomposition and program planning.

Tool-Integrated Reasoning TIR extends the role of LLMs from solvers to agents that dispatch tasks to external tools like calculators or symbolic solvers. This approach, exemplified by works such as ToRA (Gou et al., 2024a), creates a call-verify-iterate loop that enhances robustness on high-difficulty problems. Recent studies (Jin et al., 2024; Wu et al., 2024) have also demonstrated strong performance by leveraging advanced integrated environments like the GPT-4 Code Interpreter for complex problem-solving (Nguyen & Allan, 2024). The primary trade-off for TIR is the increased complexity and latency associated with tool selection and orchestration.

Ensemble-Based Reasoning To improve robustness with minimal engineering, lightweight ensemble methods aggregate multiple solution trajectories. The most prominent example is Self-Consistency, which samples multiple CoT paths and selects the answer via majority voting (Wang et al., 2023). Other works extend this by exploring more complex reasoning structures like a Tree of Thoughts (Yao et al., 2023b). These methods (Zhang et al., 2025; Yao et al., 2023a; Xia et al., 2025), however, are fundamentally forms of post-hoc selection, requiring the generation of multiple costly trajectories before aggregation and failing to identify the optimal strategy in advance.

## 3 Methodology

Figure: Figure 2: Overview of the PRISM framework. The framework consists of two stages: Offline training (top and bottom-left), where mathematical problems are solved under multiple reasoning strategies and build a dataset called MathStrat for training the Strategy Adapter; Online inference (bottom-right), where the Strategy Adapter guides adaptive routing to produce the final answer.
Refer to caption: image/framework.png

The PRISM framework is designed in two stages that decouple strategy planning from execution. The first stage involves an offline training of a Strategy Adapter (SA). This model learns to map a given problem instance to a suitability distribution over a set of reasoning strategies. The second stage is the online inference, where the prediction of SA guides an adaptive routing policy to select an execution pathway dynamically. The subsequent sections detail the mentioned stages: Section  [3.1](https://arxiv.org/html/2509.24377v1#S3.SS1) describes the formulation and training of the SA, and Section  [3.2](https://arxiv.org/html/2509.24377v1#S3.SS2) presents the adaptive routing policy used at inference.

### 3.1 Strategy Adapter

As outlined in Section [1](https://arxiv.org/html/2509.24377v1#S1), existing reasoning strategies can be broadly categorized into four paradigms: NLR, CAR, TIR, and EBR. To achieve dynamic strategy selection, we propose the strategy adaptation pipeline, implemented in two stages: (1) collection of strategy preference data, and (2) training of the strategy suitability assessment model.

Collection of Strategy Preference Data. To generate effective training signals for strategy selection, we evaluate each approach across multiple performance aspects. We construct supervision signals using three complementary dimensions that capture the essential trade-offs: (1) answer correctness, which determines whether the strategy produces the correct solution to the given mathematical problem; (2) process quality, which evaluates whether the strategy follows mathematically valid reasoning steps and avoids logical errors or redundant operations; and (3) computational efficiency, which measures whether the strategy achieves results within reasonable time and resource consumption. To implement this evaluation framework, we execute all four reasoning strategies (NLR, CAR, TIR, and EBR) on each problem instance $x$ using identical base models and decoding configurations. This yields three measurements per strategy $s$: (i) binary correctness $\mathit{corr}(s,x)\in\{0,1\}$ indicating successful problem solving; (ii) process quality $\mathit{qual}(s,x)\in[0,1]$ from an automated evaluator that penalizes invalid steps and redundant reasoning (Xia et al., 2025); and (iii) efficiency score $\mathit{eff}(s,x)\in[0,1]$ computed from raw timing and output length metrics. Specifically, we define the efficiency score $\mathit{eff}(s,x)$ as:

$$ $\mathit{eff}(s,x)=1-\tfrac{1}{2}\big(\hat{t}(s,x)+\hat{\ell}(s,x)\big),$ (1) $$

where the normalized timing and length components $\hat{t}(s,x)$ and $\hat{\ell}(s,x)$ are obtained through min-max scaling within each problem instance:

$$ $\displaystyle\hat{t}(s,x)$ $\displaystyle=\frac{t(s,x)-\min_{s^{\prime}}t(s^{\prime},x)}{\max_{s^{\prime}}t(s^{\prime},x)-\min_{s^{\prime}}t(s^{\prime},x)+\epsilon},$ (2) $\displaystyle\hat{\ell}(s,x)$ $\displaystyle=\frac{\ell(s,x)-\min_{s^{\prime}}\ell(s^{\prime},x)}{\max_{s^{\prime}}\ell(s^{\prime},x)-\min_{s^{\prime}}\ell(s^{\prime},x)+\epsilon},$ (3) $$

and $\epsilon>0$ ensures numerical stability. The three signals are aggregated using fixed weights $(w_{C},w_{Q},w_{U})$ to yield a per-strategy suitability score $\mathit{score}(s,x)$:

$$ $\mathit{score}(s,x)=w_{C}\cdot\mathit{corr}(s,x)+w_{Q}\cdot\mathit{qual}(s,x)+w_{U}\cdot\mathit{eff}(s,x).$ (4) $$

We then form a soft supervision target distribution $\mathbf{y}(x)$ by applying a temperature-scaled softmax function over the four scores within the same instance, where $y(s,x)$ represents the target probability for strategy $s$ on instance $x$:

$$ $y(s,x)=\frac{\exp\!\big(\mathit{score}(s,x)/\tau\big)}{\sum_{s^{\prime}\in\mathcal{S}}\exp\!\big(\mathit{score}(s^{\prime},x)/\tau\big)},\qquad\tau=0.5.$ (5) $$

Strategy Adapter Training. We train the Strategy Adapter $f_{\theta}$ to output logits $z_{\theta}(x)\in\mathbb{R}^{|\mathcal{S}|}$ and predict a probability distribution $p_{\theta}(x)=\mathrm{softmax}\big(z_{\theta}(x)\big)$ over strategies for each problem instance. Our training objective is designed to match the full target distribution while explicitly stabilizing the ranking of the top strategy. The primary objective minimizes the Kullback–Leibler (KL) divergence between the target distribution $\mathbf{y}(x)$ and the predicted distribution $p_{\theta}(x)$:

$$ $\mathcal{L}_{\text{dist}}(\theta)=\frac{1}{N}\sum_{i=1}^{N}\mathrm{KL}\!\left(\mathbf{y}(x_{i})\,\middle\|\,\mathbf{p}_{\theta}(x_{i})\right).$ (6) $$

To reinforce learning of the top-ranked strategy, we add an auxiliary cross-entropy loss. Let $s^{*}_{i}=\arg\max_{s\in\mathcal{S}}S(s,x_{i})$ be the best strategy for instance $x_{i}$. The auxiliary loss is:

$$ $\mathcal{L}_{\text{ord}}(\theta)=-\frac{1}{N}\sum_{i=1}^{N}\log p_{\theta,s^{*}_{i}}(x_{i}).$ (7) $$

The final loss combines these objectives:

$$ $\mathcal{L}(\theta)=\mathcal{L}_{\text{dist}}(\theta)+\lambda\mathcal{L}_{\text{ord}}(\theta),$ (8) $$

where $\lambda$ is a hyperparameter that balances the two objectives. This combined distributional and ranking-aware objective encourages the model to capture the relative suitability of strategy families, which guides the inference-time routing policy. We implement the Strategy Adapter as a lightweight language model (e.g., 1.5B parameters) trained on our curated dataset of approximately 13,000 problem instances with multi-strategy performance evaluations. Upon completion of training, the adapter demonstrates effective suitability assessment capabilities, with a representative example of its prediction behavior provided in Appendix [9](https://arxiv.org/html/2509.24377v1#A1.F9).

### 3.2 Adaptive Routing Policy at Inference

The Strategy Adapter produces a probability distribution $p_{\theta}(x)$ over strategies for each problem instance $x$. Simply selecting the highest-probability strategy, however, would result in uniform single-strategy execution regardless of prediction confidence (as illustrated in Figure [1](https://arxiv.org/html/2509.24377v1#S1.F1)). This approach fails to exploit opportunities for computational efficiency when predictions are highly confident or for enhanced robustness when predictions are uncertain.

Our adaptive routing policy interprets the predictor output through a confidence-based framework that dynamically selects among three execution modes: *Confident*, *Deliberative*, and *Exploratory* routing. The mode selection depends on two calibrated thresholds: a confidence threshold $\tau_{c}$ and an ambiguity margin $\tau_{a}$, which are optimized through grid search on a validation set (see Appendix [A.3](https://arxiv.org/html/2509.24377v1#A1.SS3) for details), applied to the top two predicted probabilities $p_{\max}$ and $p_{2\text{nd}}$. Confident Routing ($p_{\max}\geq\tau_{c}$ and $(p_{\max}-p_{2\text{nd}})\geq\tau_{a}$) executes only the single best-ranked strategy when the predictor exhibits high confidence with a clear preference. Deliberative Routing ($p_{\max}\geq\tau_{c}$ and $(p_{\max}-p_{2\text{nd}})<\tau_{a}$) executes the top two strategies when confidence is high but rankings are close, aggregating results through majority voting. Exploratory Routing ($p_{\max}<\tau_{c}$) executes all available strategies when predictor confidence is insufficient, again using majority voting for final answer selection. The routing mode distribution under different threshold configurations on this validation set is analyzed in Figure [4](https://arxiv.org/html/2509.24377v1#S4.F4) (left), demonstrating that lower confidence thresholds lead to increased confident routing (single-strategy execution) while higher thresholds favor more exploratory multi-strategy approaches. For multi-strategy modes, answers undergo standardization to normalize numerical formats before voting, with ties resolved by selecting the strategy with highest predicted probability. This mechanism provides principled computational resource allocation without requiring external verification components.

## 4 Experiment

### 4.1 Setup

#### Datasets and Baselines

We use as diverse mathematical datasets as possible for experiments. In addition to the widely used MATH (Hendrycks et al., 2021) and GSM8K (Cobbe et al., 2021) dataset, we also adopt AQUA-RAT (Ling et al., 2017), SVAMP (Patel et al., 2021) and ASDiv (Miao et al., 2020). These datasets cover multiple fields of mathematics, such as elementary arithmetic problems, mathematical algebra, inferential counting, and probability number theory. They also span a wide range of difficulty levels, including simple elementary school math problems, intermediate-level questions, and even Olympiad-style competition problems. We select large language models from three series. Our experiments use Qwen2.5-Math-7B (Yang et al., 2024), Deepseek-math-7b-v1 (Lu et al., 2024a), and Llama-3-8B to conduct thorough evaluations. For evaluation metrics, we report Pass@k accuracy, where a problem is considered solved if the correct answer appears in the top-$k$ generated solutions. Specifically, Pass@1 reflects single-shot correctness, while Pass@5 captures the probability of producing at least one correct solution among five independent generations.

#### Reasoning Approaches

As mentioned in previous studies, our experiment also involves four other mathematical reasoning approaches like CoT (Wei et al., 2022a), PAL (Gao et al., 2023a), ToRA (Zhang et al., 2023), Hybrid (Yue et al., 2023). Chain-of-Thought (CoT) prompting is a technique designed to elicit more robust reasoning from LLMs by encouraging them to generate a series of intermediate, step-by-step rationales before concluding with a final answer. Program-Aided Language Models (PAL) introduce a neuro-symbolic approach that offloads the reasoning and calculation logic to an external tool. Tool-Augmented Reasoning Agent (ToRA) can interleave natural language reasoning steps with calls to different tools, such as a calculator, a symbolic solver, or retrieval APIs. Hybrid approaches aim to combine the strengths of different reasoning paradigms to achieve superior performance and robustness.

### 4.2 Main Results

Table [1](https://arxiv.org/html/2509.24377v1#S4.T1) shows performance across three base models and five mathematical reasoning benchmarks.
PRISM achieves average improvements of 0.9% on Qwen2.5-Math-7B, 2.9% on Deepseek-math-7b-v1, and 7.6% on Llama-3-8B over the best single strategies.
The inverse relationship between relative improvement and base model capability suggests that strategic routing provides greater value when addressing model limitations.
The results confirm our central observation that no single strategy dominates across all benchmarks—while ToRA excels on MATH500, PAL leads on GSM8K for Qwen (95.3%), and performance varies dramatically across datasets.
PRISM effectively handles this heterogeneity through adaptive strategy selection.
The method substantially outperforms the Hybrid baseline, particularly on complex reasoning tasks like AQUA-RAT, demonstrating that pre-execution routing is more effective than post-hoc strategy aggregation.
Individual strategies exhibit high variance (PAL ranges from 13.5% to 95.3%), while PRISM maintains consistent performance across all test conditions.

Table: Table 1: Performance comparison of different mathematical reasoning strategies across three base language models and five datasets. CoT refers to Chain-of-Thought reasoning, PAL to Program-Aided Language models, ToRA to Tool-integrated Reasoning Agent, and Hybrid to ensemble-based approaches. PRISM represents our proposed adaptive routing framework. Pass@k denotes the percentage of problems for which at least one correct solution appears in the top-k generated outputs. Blue highlighting indicates the best performance for each model-dataset combination.

### 4.3 Ablation Study On Routing Components

To understand the contribution of each routing component, we conducted progressive ablation experiments across GSM8K, MATH500, and Hungarian Math datasets. As detailed in [2](https://arxiv.org/html/2509.24377v1#S4.T2), adding confident routing shows mixed results—76.0% on GSM8K (below the 78.1% CoT baseline) but improvements on MATH500 (28.6% vs 21.2%) and Hungarian Math (50.0% vs 40.6%). Incorporating deliberative routing provides continued gains on MATH500 (32.2%) with variable performance elsewhere. The complete PRISM system achieves substantial improvements across all datasets (86.7%, 46.2%, and 53.1% respectively), with dramatic jumps from the previous configuration demonstrating that the full adaptive routing policy is essential for optimal performance. These results validate that individual routing modes provide limited benefits, while the intelligent coordination of all components through problem-aware strategy selection delivers significant performance gains.

**Table 2: Ablation study of adaptive routing components using the Qwen2.5-Math-7B model.**
| Setting | GSM8K | MATH500 | Hungarian Math |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| pass@1 | pass@5 | pass@1 | pass@5 | pass@1 | pass@5 |  |
| CoT baseline | 78.1 | 93.5 | 21.2 | 50.8 | 40.6 | 56.3 |
| Confident | 76.0 | 95.0 | 28.6 | 56.1 | 50.0 | 62.5 |
| Confident + Deliberative | 75.8 | 95.6 | 32.2 | 57.3 | 43.8 | 50.0 |
| PRISM | 86.7 | 96.0 | 46.2 | 64.4 | 53.1 | 71.9 |

### 4.4 Performance-Efficiency Trade-Off

To examine the computational efficiency of our adaptive routing approach, we measured performance and resource consumption across different framework configurations, as shown in Figure [3](https://arxiv.org/html/2509.24377v1#S4.F3). The baseline CoT approach achieves 9.5% Pass@1 accuracy with 5,200ms inference time and 45-token average output length. Adding the Strategy Adapter with confident routing (SA+Conf.) shows minimal performance improvement to 10.0% but increases computational cost to 6,300ms and 50 tokens. The combination of confident and deliberative routing (SA+Conf.+Delib.) achieves 17.8% accuracy while maintaining similar efficiency profiles at 6000ms and 80 tokens. The complete PRISM system demonstrates substantial performance gains, reaching 33.3% Pass@1 accuracy while achieving better efficiency than intermediate configurations at 5,600ms inference time and 85 tokens output length. This efficiency-performance trade-off reveals that the full adaptive routing policy not only improves accuracy but also optimizes resource utilization by intelligently selecting execution pathways. The results indicate that the Strategy Adapter alone provides limited benefits, but when combined with the complete adaptive routing mechanism, it enables significant performance improvements while maintaining computational efficiency. This validates our design choice of integrating prediction-guided strategy selection with dynamic execution pathways rather than relying on individual components in isolation.

Figure: Figure 3: Analysis of PRISM framework components across three key metrics. (a) Pass@1 accuracy shows substantial gains with the full system. (b) Inference time and (c) output length demonstrate that PRISM achieves higher performance with better or comparable computational efficiency than intermediate configurations.
Refer to caption: image/6.png

### 4.5 Scalability Analysis

To evaluate the scalability of our approach, we conducted experiments across Qwen2.5 models ranging from 1.5B to 72B parameters on GSM8K and MATH500 benchmarks, using Qwen2.5 7B with chain-of-thought prompting as our baseline. As illustrated in Figure [4](https://arxiv.org/html/2509.24377v1#S4.F4) (right), PRISM demonstrates consistent improvements over the baseline across all model scales, achieving accuracy from 74.2% to 95.2% on GSM8K and 32.3% to 78.4% on MATH500. The framework exhibits distinct scaling patterns across benchmarks: steady improvements on GSM8K with notable gains from 7B to 32B parameters, and more dramatic scaling effects on MATH500 where performance nearly doubles from smallest to largest models. These scaling results validate that adaptive strategy selection provides robust benefits across different model capacities. The sustained improvements across parameter scales demonstrate that the framework generalizes effectively and does not depend on specific model characteristics to achieve performance gains. Importantly, since PRISM operates as a training-free approach that works purely through inference-time strategy selection, it can be readily applied to any pre-trained model without requiring additional fine-tuning or domain-specific training, making it broadly applicable across different model families and computational budgets.

Figure: Figure 4: Left: Routing mode distribution analysis across different confidence threshold configurations. The stacked bars show the percentage of problems routed to each execution mode (Confident, Deliberative, Exploratory) for varying $\tau_{c}$ values while keeping $\tau_{a}=0.08$. Right: Scalability of PRISM across Qwen2.5 models of varying sizes on GSM8K and MATH500 benchmarks. Dotted lines indicate the baseline performance of Qwen2.5-7B with standard chain-of-thought prompting.
Refer to caption: image/combine3.png

### 4.6 Strategy Adapter Behavior Analysis

We analyze the prediction behavior patterns of our Strategy Adapter across different mathematical reasoning datasets to validate its learned strategy selection characteristics. This analysis examines both the confidence levels in predictions and the competitive landscape among strategies. Figure [5](https://arxiv.org/html/2509.24377v1#S4.F5) presents the distributions of prediction confidence ($p_{\max}$) and strategy competition gaps ($p_{\max}-p_{\text{2nd}}$) across four mathematical reasoning datasets. The results reveal several important patterns that validate our framework design. First, the SA exhibits prediction confidence patterns that correlate with dataset complexity. On MATH500, which contains competition-level mathematical problems, the predictor shows notably conservative behavior with prediction confidence ($p_{\max}$) concentrated in the 0.0 to 0.2 range. This low confidence reflects both the inherent difficulty of these problems and the SA’s learned caution when dealing with competition-level mathematics, where strategy effectiveness is less predictable.

In contrast, datasets containing more elementary mathematical problems show progressively higher prediction confidence. GSM8K demonstrates moderate confidence levels with $p_{\max}$ distributed across $0.2\sim 0.6$, while ASDiv and SVAMP exhibit relatively higher confidence with peaks around $0.3\sim 0.4$. This graduated confidence pattern indicates that the SA has successfully learned to associate problem complexity with prediction uncertainty, demonstrating sophisticated meta-reasoning about strategy applicability. The adaptive confidence calibration also suggests that the SA effectively captures the inherent variability in strategy effectiveness across different mathematical domains, with higher uncertainty appropriately assigned to problems where multiple strategies might yield similar performance. Furthermore, the distribution shapes themselves provide insight into the SA’s decision-making process: sharp peaks indicate clear strategy preferences for certain problem types, while flatter distributions suggest scenarios where multiple strategies remain viable options. The strategy competition analysis reveals consistently small gaps between top strategies across all datasets. This competitive landscape validates our adaptive routing design rationale: the narrow margins between strategy preferences require nuanced confidence-based decision making rather than simple winner-take-all selection. Additionally, we provide a detailed analysis of strategy performance patterns and inter-strategy correlations across datasets in Appendix [A.4](https://arxiv.org/html/2509.24377v1#A1.SS4).

Figure: Figure 5: Strategy Adapter behavior across mathematical reasoning datasets. Top row shows prediction confidence ($p_{\max}$) distributions, bottom row shows strategy competition gaps ($p_{\max}-p_{2\text{nd}}$). The SA exhibits dataset-appropriate confidence levels: conservative predictions on competition problems (MATH500) and higher confidence on elementary problems (ASDiv, SVAMP), while maintaining competitive strategy landscapes across all datasets.
Refer to caption: image/pmax.png

## 5 Conclusion

We introduce a problem-aware strategy routing framework termed PRISM, which decouples mathematical reasoning into strategy planning and targeted execution. Specifically, it leverages a multi-strategy performance profiling mechanism to curate a 13K strategy preference dataset MathStrat. Then, a strategy adaptor is trained on this dataset to perform policy routing for the given problem at inference time. Extensive experiments with three language models across five datasets, combined with comprehensive ablation studies and efficiency analysis, demonstrate the effectiveness, superiority, and scalability of PRISM.

## Acknowledgments and Disclosure of Funding

This work was supported in part by the National Key Research and Development Program of China (2022YFC3303600), the National Natural Science Foundation of China (62137002, 62306229, 62477037, 62293553), the Natural Science Basic Research Program of Shaanxi (2023-JC-YB-593), the Key Research and Development Program of Shaanxi (2024GX-ZDCYL-02-12), the Youth Innovation Team of Shaanxi Universities “Multi-modal Data Mining and Fusion”, the Shaanxi Undergraduate and Higher Education Teaching Reform Research Program (23BY195), the Youth Talent Support Program of Shaanxi Science and Technology Association (20240113), and the China Postdoctoral Science Foundation (2024M752585, 2025T180425).

## References

- Chen et al. (2023)
Wenhu Chen, Xueguang Zhao, Chang Shu, Hongjin Chen, Zeqiu Lin, Yulei Wang, William Yang Wang, et al.
Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.
*Transactions on Machine Learning Research*, 2023.
- Cobbe et al. (2021)
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman.
Training verifiers to solve math word problems.
*arXiv preprint arXiv:2110.14168*, 2021.
- Didolkar et al. (2024)
Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy Lillicrap, Danilo Rezende, Yoshua Bengio, Michael Mozer, and Sanjeev Arora.
Metacognitive capabilities of llms: An exploration in mathematical problem solving.
In *NeurIPS*, pp. 19783–19812, 2024.
- Gao et al. (2023a)
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig.
Pal: Program-aided language models.
In *ICML*, pp. 10764–10799, 2023a.
- Gao et al. (2023b)
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, Jamie Callan, and Graham Neubig.
Pal: Program-aided language models.
In *ICML*, pp. 10764–10799, 2023b.
- Gou et al. (2024a)
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen.
Tora: A tool-integrated reasoning agent for mathematical problem solving.
In *ICLR*, 2024a.
- Gou et al. (2024b)
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al.
Tora: A tool-integrated reasoning agent for mathematical problem solving.
In *ICLR*, 2024b.
- Hendrycks et al. (2021)
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.
Measuring mathematical problem solving with the math dataset.
In *NeurIPS*, 2021.
- Jin et al. (2024)
Aili Jin, Zhi-Yao Tang, Hong-Yu Zhang, Chen-Yu Wang, and Ming Wan.
T-rex: A new frontier for tool-augmented reasoning with large language models.
In *ICLR*, 2024.
- Li et al. (2024)
Zenan Li, Zhi Zhou, Yuan Yao, Yu-Feng Li, Chun Cao, Fan Yang, Xian Zhang, and Xiaoxing Ma.
Neuro-symbolic data generation for math reasoning.
In *NeurIPS*, pp. 23488–23515, 2024.
- Ling et al. (2017)
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom.
Program induction by rationale generation: Learning to solve and explain algebraic word problems.
In *ACL*, pp. 158–167, 2017.
- Lu et al. (2024a)
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan.
Deepseek-vl: Towards real-world vision-language understanding.
*arXiv preprint arXiv:2403.05525*, 2024a.
- Lu et al. (2024b)
Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li.
Mathgenie: Generating synthetic data with question back-translation for enhancing mathematical reasoning of llms.
In *ACL*, 2024b.
- Luo et al. (2025)
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang.
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.
In *ICLR*, 2025.
- Ma et al. (2025a)
Jie Ma, Zhitao Gao, Qi Chai, Wangchun Sun, Pinghui Wang, Hongbin Pei, Jing Tao, Lingyun Song, Jun Liu, Chen Zhang, et al.
Debate on graph: a flexible and reliable reasoning framework for large language models.
In *AAAI*, pp. 24768–24776, 2025a.
- Ma et al. (2025b)
Jie Ma, Ning Qu, Zhitao Gao, Rui Xing, Jun Liu, Hongbin Pei, Jiang Xie, Linyun Song, Pinghui Wang, Jing Tao, et al.
Deliberation on priors: Trustworthy reasoning of large language models on knowledge graphs.
*arXiv preprint arXiv:2505.15210*, 2025b.
- Miao et al. (2020)
Shen-Yun Miao, Chao-Chun Liang, and Keh-Yih Su.
A diverse corpus for evaluating and developing english math word problem solvers.
In *ACL*, pp. 975–984, 2020.
- Nguyen & Allan (2024)
Ha Nguyen and Vicki Allan.
Using gpt-4 to provide tiered, formative code feedback.
In *Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1*, pp. 958–964, 2024.
- Patel et al. (2021)
Arkil Patel, Satwik Bhattamishra, and Navin Goyal.
Are nlp models really able to solve simple math word problems?
*arXiv preprint arXiv:2103.07191*, 2021.
- Ranaldi et al. (2024)
Leonardo Ranaldi, Giulia Pucci, Barry Haddow, and Alexandra Birch.
Empowering multi-step reasoning across languages via program-aided language models.
In *EMNLP*, 2024.
- Trivedi et al. (2025)
Prashant Trivedi, Souradip Chakraborty, Avinash Reddy, Vaneet Aggarwal, Amrit Singh Bedi, and George K. Atia.
Align-pro: A principled approach to prompt optimization for llm alignment.
In *AAAI*, pp. 27653–27661, 2025.
- Wang et al. (2024)
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li.
Measuring multimodal mathematical reasoning with math-vision dataset.
In *NeurIPS*, pp. 95095–95169, 2024.
- Wang et al. (2023)
Xuezhi Wang, Jason Wei, Dale Schuurmans, Ed H. Chi, Quoc V. Le, and Denny Zhou.
Self-consistency improves chain of thought reasoning in language models.
In *ICLR*, 2023.
- Wei et al. (2022a)
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou.
Chain-of-thought prompting elicits reasoning in large language models.
In *NeurIPS*, 2022a.
- Wei et al. (2022b)
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou.
Chain-of-thought prompting elicits reasoning in large language models.
In *NeurIPS*, pp. 24824–24837, 2022b.
- Wu et al. (2024)
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, and Li Jiang.
Autogen: Enabling next-gen llm applications via multi-agent conversation.
In *ICLR*, 2024.
- Xia et al. (2025)
Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu.
Evaluating mathematical reasoning beyond accuracy.
In *AAAI*, 2025.
- Xie et al. (2025)
Wenjing Xie, Xiaobo Liang, Juntao Li, Wanfu Wang, Kehai Chen, Qiaoming Zhu, and Min Zhang.
From awareness to adaptability: Enhancing tool utilization for scientific reasoning.
In *ACL*, 2025.
- Xin et al. (2025)
Huajian Xin, ZZ Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, et al.
Deepseek-prover-v1.5: Harnessing proof assistant feedback for reinforcement learning and monte-carlo tree search.
In *The Thirteenth International Conference on Learning Representations*, 2025.
- Yang et al. (2024)
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang.
Qwen2.5-math technical report: Toward mathematical expert model via self-improvement.
*arXiv preprint arXiv:2409.12122*, 2024.
- Yao et al. (2023a)
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan.
Tree of thoughts: Deliberate problem solving with large language models.
In *NeurIPS*, pp. 11809–11822, 2023a.
- Yao et al. (2023b)
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.
Tree of thoughts: Deliberate problem solving with large language models.
In *NeurIPS*, pp. 11809–11822, 2023b.
- Ye et al. (2024)
Jiacheng Ye, Shansan Gong, Liheng Chen, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Xin Jiang, Zhenguo Li, Wei Bi, and Lingpeng Kong.
Diffusion of thought: Chain-of-thought reasoning in diffusion language models.
In *NeurIPS*, pp. 105345–105374, 2024.
- Yu et al. (2024)
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu.
Metamath: Bootstrap your own mathematical questions for large language models.
In *ICLR*, 2024.
- Yue et al. (2023)
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen.
Mammoth: Building math generalist models through hybrid instruction tuning.
*arXiv preprint arXiv:2309.05653*, 2023.
- Zhang et al. (2023)
Beichen Zhang, Kun Zhou, Xilin Wei, Xin Zhao, Jing Sha, Shijin Wang, and Ji-Rong Wen.
Evaluating and improving tool-augmented computation-intensive math reasoning.
In *NeurIPS*, pp. 23570–23589, 2023.
- Zhang et al. (2025)
Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, Wanli Ouyang, and Dongzhan Zhou.
Llama-berry: Pairwise optimization for olympiad-level mathematical reasoning via o1-like monte carlo tree search.
In *NAACL*, pp. 7315–7337, 2025.
- Zhang & Xiong (2025)
Shaowei Zhang and Deyi Xiong.
Backmath: Towards backward reasoning for solving math problems step by step.
In *COLING*, pp. 466–482, 2025.
- Zhang et al. (2024)
Zhehao Zhang, Jiaao Chen, and Diyi Yang.
Darg: Dynamic evaluation of large language models via adaptive reasoning graph.
In *NeurIPS*, pp. 135904–135942, 2024.

## Appendix A Appendix

### A.1 Adaptive Strategy Routing algorithm

This section provides the detailed pseudo-code for the PRISM adaptive routing policy, as referenced in Section [3.2](https://arxiv.org/html/2509.24377v1#S3.SS2).

Figure: Algorithm 1 ADAPTIVE ROUTING POLICY AT INFERENCE

### A.2 Preference Data Example

Figure [6](https://arxiv.org/html/2509.24377v1#A1.F6) presents a representative case of our multi-strategy performance evaluation process for collecting training data. The case shows a trigonometric function analysis problem where we execute all four reasoning strategies and collect comprehensive performance metrics.

The data collection captures three complementary dimensions as described in our methodology: answer correctness (validity scores), process quality (redundancy measures), and computational efficiency (execution time and output length). As illustrated in this case, different strategies exhibit distinct performance profiles: NLR achieves moderate validity (0.23) but suffers from high redundancy (0.68), while TIR demonstrates better process quality with higher validity (0.29) and lower redundancy (0.72). CAR shows the poorest validity (0.18) with the highest redundancy (0.73), and EBR balances the highest validity (0.35) with acceptable redundancy (0.66).

The condensed output summaries reveal how each strategy approaches the same trigonometric problem differently: NLR relies on numerical approximation methods, CAR attempts algebraic simplification, TIR uses analytical identities, and EBR combines multiple approaches. This diversity in solution paths, combined with varying performance across the three evaluation dimensions, provides rich supervision signals that enable our Strategy Adapter to learn nuanced associations between problem characteristics and strategy effectiveness. The multi-faceted evaluation ensures that our training data captures the essential trade-offs between correctness, reasoning quality, and computational cost.

Figure: Figure 6: Strategy preference data collection showing multi-strategy performance evaluation for a trigonometric function problem. Each strategy exhibits distinct profiles across the three evaluation dimensions: correctness, process quality, and computational efficiency.
Refer to caption: image/case7.png

### A.3 Threshold Parameter Optimization

To determine the optimal confidence threshold $\tau_{c}$ and ambiguity margin $\tau_{a}$ for our adaptive routing policy, we conducted a comprehensive grid search on a validation set comprising 200 problems sampled from MATH, GSM8K, AQUA-RAT, SVAMP, and ASDiv datasets to ensure coverage across different mathematical domains and difficulty levels. We evaluated $\tau_{c}\in[0.1,0.7]$ and $\tau_{a}\in[0.02,0.20]$ with step sizes of 0.05 and 0.01 respectively. Figure [7](https://arxiv.org/html/2509.24377v1#A1.F7) shows the parameter optimization results across the two-dimensional parameter space. The contour plot reveals a well-defined optimal region at $\tau_{c}=0.4$ and $\tau_{a}=0.08$, achieving 78.0% Pass@1 accuracy on the validation set. The performance landscape exhibits several notable characteristics:

Confidence Threshold Sensitivity: The $\tau_{c}$ parameter shows an inverted-U relationship with performance (Figure [7](https://arxiv.org/html/2509.24377v1#A1.F7), top right panel). Very low confidence thresholds ($\tau_{c}<0.2$) result in over-conservative routing that fails to leverage high-confidence predictions effectively, achieving only 65% accuracy. Conversely, excessively high thresholds ($\tau_{c}>0.5$) force the system into single-strategy execution even for uncertain predictions, degrading performance to 71%.

Ambiguity Margin Sensitivity: The $\tau_{a}$ parameter demonstrates a sharp optimum (Figure [7](https://arxiv.org/html/2509.24377v1#A1.F7), bottom right panel). The optimal value of 0.08 creates an appropriate balance for distinguishing competitive strategy scenarios. Lower values ($\tau_{a}<0.06$) cause excessive deliberative routing even when strategy preferences are clear, while higher values ($\tau_{a}>0.12$) prevent beneficial dual-strategy verification in genuinely competitive cases.

The relatively narrow optimal region indicates that careful parameter tuning is essential for achieving peak performance. However, the clear convex structure around the optimum suggests stable convergence during hyperparameter search. We use these validated parameters ($\tau_{c}=0.4,\tau_{a}=0.08$) across all experimental settings to ensure fair comparison with baseline methods.

Figure: Figure 7: Threshold parameter optimization on validation set. Left: Contour plot showing accuracy across the $\tau_{c}$-$\tau_{a}$ parameter space with optimal point marked by red star. Right: Sensitivity analysis showing accuracy curves for individual parameters while holding the other at optimal value. The validation set consists of 200 problems sampled across MATH, GSM8K, AQUA-RAT, SVAMP, and ASDiv to ensure diverse difficulty coverage.
Refer to caption: image/80.png

### A.4 Strategy Performance and Correlation Analysis

Figure [8](https://arxiv.org/html/2509.24377v1#A1.F8) presents the mean performance scores and inter-strategy correlations across four mathematical reasoning datasets. The left panel shows that all strategies achieve similar average suitability scores (ranging from 0.18 to 0.31), with no single strategy demonstrating clear dominance across datasets. The right panel displays correlation matrices revealing predominantly low or negative correlations between strategies, indicating complementary rather than redundant capabilities. Notably, TIR exhibits consistent negative correlations with other strategies across most datasets, suggesting its specialized applicability to distinct problem characteristics. These patterns validate the necessity of adaptive strategy selection, as the low inter-strategy correlations demonstrate that different approaches excel on different problem subsets.

Figure: Figure 8: Strategy performance and correlation analysis. Left: Mean suitability scores by strategy and dataset. Right: Inter-strategy correlation matrices for each dataset. Low correlations indicate complementary strategy capabilities.
Refer to caption: image/heatmaps.png

### A.5 Case Study: Strategy Adapter Prediction

Figure: Figure 9: Case study showing Strategy Adapter suitability scores and corresponding strategy outcomes for a logarithmic geometry problem. Higher suitability scores correlate with successful problem-solving performance.
Refer to caption: image/case9.png

Figure [9](https://arxiv.org/html/2509.24377v1#A1.F9) presents a representative example demonstrating how our Strategy Adapter evaluates different reasoning approaches for a logarithmic geometry problem. The problem requires finding the x-coordinate where a horizontal line intersects the curve $f(x)=\ln x$, involving both coordinate geometry concepts and logarithmic calculations.

The Strategy Adapter assigns suitability scores that align well with actual strategy performance: TIR receives the highest score (0.63) and successfully solves the problem through tool-assisted computation, while NLR and CAR receive lower scores (0.28 and 0.16 respectively) and both fail to produce correct solutions. EBR achieves a moderate score (0.58) and succeeds through ensemble reasoning. This case exemplifies how the adapter learns to associate problem characteristics—such as the need for precise numerical computation in logarithmic contexts—with appropriate reasoning strategies.

The correlation between prediction scores and actual outcomes validates our approach of using suitability scores to guide adaptive routing decisions, demonstrating that higher Strategy Adapter scores generally correspond to better strategy performance on specific problem instances.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: - How is “process quality” computed by the automated evaluator? This step seems difficult and potentially subjective to the model used.  

- What datasets are used to construct MathStrat? Is the Strategy Adapter tested on datasets that were not included in training? How much overlap exists between the MathStrat construction data and the evaluation benchmarks?
soundness: 2
strengths: - The idea of training a lightweight Strategy Adapter that enables a dynamic and adaptive routing policy for reasoning strategy is interesting.
- The empirical results are effective, demonstrating consistent gains across tasks.
- The ablation studies are extensive and useful, carefully evaluating the effects of routing modes, efficiency, and scalability.
summary: The paper shows that there is no single reasoning strategy that consistently performs well both efficiently and effectively across mathematical reasoning tasks. It proposes to train a Strategy Adapter on a multi-strategy preference dataset constructed based on correctness, process quality, and computational efficiency. The lightweight adapter dynamically routes between confident, deliberative, and exploratory modes at test time to select the reasoning strategy to explore for each question instance. Empirical analysis across five mathematical reasoning benchmarks shows consistent improvements, particularly when the model scale is small.
weaknesses: - Related work discussion is limited. The Strategy Adapter sounds similar to a MOE gating network, and other routing or adaptive ideas have been explored in prior work on tool use. These connections are not sufficiently compared or discussed.
- The MathStrat dataset includes only four reasoning strategies, and the adapter is trained to route among them dynamically. This design still limits the reasoning space to those four strategies and is not easily extendable to broader reasoning paradigms without reconstructing the dataset and retraining the adapter from scratch.
- The main results (Table 1) compare PRISM with four other strategies in isolation. A stronger baseline that jointly utilizes all four reasoning strategies is desirable. While the “Hybrid” baseline combines CoT and PoT, it does not encompass or compare against a setting that integrates all strategies. Including a model that executes all strategies simultaneously, even without adaptive selection, would help demonstrate the effectiveness of PRISM in a fairer manner.

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: Refer to Fig. (3), I am curious about why PRISM generates more tokens (see subfigure c) but it takes relatively less inference time (subfigure b).
soundness: 3
strengths: 1. The paper targets a central limitation in LLM reasoning: static inference paradigms that cannot adjust reasoning depth or modality to instance-level complexity. The problem is well-motivated by Figure 1, which empirically demonstrates cross-strategy variance in both accuracy and efficiency. This finding aligns with emerging work on meta-reasoning (Didolkar et al., 2024).

2. Section 4.4 (Fig. 3) convincingly shows that PRISM improves Pass@1 accuracy while reducing or maintaining inference cost compared to hybrid baselines. The confident/deliberative/exploratory routing design (Section 3.2) provides a clear operational mechanism for adaptive compute allocation without external verifiers.
summary: This paper presents a technically sound, well-motivated, and methodologically clean contribution to inference-time reasoning optimization for LLMs. The formulation of mathematical reasoning as problem-aware strategy routing is conceptually novel and bridges ideas from meta-reasoning and tool orchestration. However, several points need clarification: the construction and bias of the MathStrat dataset, the scalability of the adapter to unseen strategies, and the robustness of routing thresholds (τ_c, τ_a) across domains. Moreover, while empirical improvements are consistent, the reported gains (≈1–7%) may be modest relative to the added system complexity
weaknesses: 1. MathStrat combines correctness, quality, and efficiency via fixed weights (Eq. 4), but the weighting scheme (w_C, w_Q, w_U) and its sensitivity are not justified or ablated. Potential annotation bias may transfer to adapter predictions.

2. Only four strategy families (NLR, CAR, TIR, EBR) are considered. It is unclear how PRISM generalizes to newly emerging paradigms (e.g., tree-of-thought, verifier-augmented reasoning).

3. The confidence and ambiguity thresholds (τ_c = 0.4, τ_a = 0.08) are globally fixed (Appendix A.3); more studies are needed to verify their stability across datasets or model sizes.

4. The paper does not detail where PRISM misroutes (e.g., confident errors vs. unnecessary exploration) or how confidence calibration could fail.

5. The adaptor is only trained via supervised finetuning (SFT). I expect an effective reinforcement learning (RL) method to optimize the adaptor on new/unseen examples, improving its generalization.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: **[Q1]** There appears to be an inconsistency in the baseline results reported in the paper. In Table 1, under the CoT strategy, Qwen2.5-Math-7B achieves 78.1 on GSM8K and 21.2 on MATH500. However, according to the official Qwen2.5-Math report [2], the Qwen2.5-Math-7B model reaches 91.6 (GSM8K) and 55.4 (MATH) in the few-shot setting, while the Qwen2.5-Math-7B-Instruct model achieves 95.2 (GSM8K) and 83.6 (MATH) in the zero-shot setting. Could the authors provide a reasonable explanation for this discrepancy?

**[Q2]** Could the authors provide a detailed description of how the inference time is calculated? I could not find this information in the paper, and it is critical for assessing the validity of the claimed efficiency improvements.

Ref:  
[2] Yang et al. Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement. arXiv 2409.12122.
soundness: 2
strengths: **[S1]** The paper is well-written, with clear methodological descriptions. The proposed approach is easy to follow, and the authors provide code for reproducibility.  
**[S2]** The paper addresses an important problem: how to enable models to adopt different reasoning strategies depending on the complexity of a given problem. For simple problems, the model can rely on its internal knowledge, while for more difficult ones, it needs to leverage code and external tools for problem-solving.  
**[S3]** The paper presents a comprehensive set of experiments, including detailed ablation studies.
summary: This paper proposes PRISM, a framework that enables large language models to dynamically apply multiple strategies to solve mathematical problems. The proposed method trains a strategy adapter that, before the model begins problem solving, selects one or more strategies from Natural Language Reasoning (NLR), Code-Augmented Reasoning (CAR), Tool-Integrated Reasoning (TIR), and Ensemble-Based Reasoning (EBR). This allows the model to adopt the most suitable strategy for a given problem and thereby improve both problem-solving accuracy and efficiency.

To support PRISM, the paper introduces a method for constructing training data for the adapter and develops the MathStrat dataset specifically for this purpose. Experimental results show that models enhanced with PRISM outperform multiple baseline approaches.
weaknesses: **[W1] Methodology:** The paper trains a separate Strategy Adapter (SA) independent of the LLM to select problem-solving strategies. However, the authors do not explain why a separate SA is necessary, rather than enabling the LLM itself to dynamically select the appropriate reasoning strategy. If the SA can effectively choose suitable strategies, the LLM should in principle be capable of learning this ability as well.

**[W2] Originality:** Prior work [1] has already explored dynamic selection among multiple reasoning strategies, yet the authors do not provide comparisons with such related studies—neither in terms of methodological design nor empirical performance.

**[W3] Fairness of Experimental Comparison:** It is unclear whether the comparison between PRISM and CoT is fair. CoT performs only a single inference, whereas PRISM may conduct multiple inference rounds followed by majority voting (for instance, if the SA selects Exploratory Routing, the model performs four different reasoning attempts). Such a setup may lead to an unfair advantage for PRISM.

**[W4] Robustness:** The construction of the MathStrat dataset involves a large number of hyperparameters, yet the authors do not provide a systematic analysis of the model’s sensitivity to these hyperparameters.

Ref:  
[1] Xu et al. Teaching llms according to their aptitude: Adaptive reasoning for mathematical problem solving. arXiv 2502.12022.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. Line 191: "existing reasoning strategies can be broadly categorized into four paradigms," if you are saying "broadly," did you routing method covers all cases? i.e., you do have the probability that you miss one important reasoning method, how can you prove your categorization is complete? 
2. Line 198: "(2) process quality," how do you evaluate "process quality?" This is unclear. Are you only checking if the proof is logically coherent? 
3. Line 199: "(3) computational efficiency," how do you quantify this metric? By tokens during computation? execution (thinking) time? model size? Need to be more specific here.
4. eq 7, a bit confusing, can you explain what this loss is doing? enforcing the top-1 selection? 
5. line 263, I might misunderstanding something. How you can do majority voting for 2 strategies?
soundness: 2
strengths: 1. Routing-based method seems intuitive 
2. Optimized the router with self-created dataset 
3. Did ablation on adaptive routing component
summary: This paper introduces PRISM, a routing-based framework that (1) categorize each math problem with "soft labels," and (2) execute some prompting strategies of the router. They've shown improvements on some math dataset, and they conducted ablations on components of adaptive routing.
weaknesses: 1. This method, from my perspective, is somehow similar to [1]. In math reasoning, what HybridMind did is also decompose (route) the problem into the "correct" or "groundtruth" one. The difference, from my end, is they train a hard classifier using CE loss, while you train a soft labler using KL + CE loss. But you lack proper citation in your related work. I anticipate to see more comparison with similar-idea routing based work. 
2. Similar to Question 1, how can you make sure the the categorized reasoning types are complete? 
3. I cannot fully be convinced that using raw timing in efficiency score, using output length (in tokens) seems more natural. For example, if I want to use this method with some proprietary models, are you going to waiting time as a training signal in this case?
4. Any specific reasonings you choose these three models in your evaluation? What's the performance of some proprietary models, say an old one, GPT 3.5 or GPT 4.
5. What's the major advantage of using KL divergence during router training compared to classification / multi-class classification? 
6. Is this method generalizable? Have you tried to adapt this method to other domain, say logical reasoning and scientific reasoning? My impression for these two kind of reasonings are somehow similar to math, but maybe different from some angles. But it would be interesting to see if your method would success/fail/partial in other domains. 

[1] Han, S., Liu, T., Li, C., Xiong, X., & Cohan, A. (2024). HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning. arXiv preprint arXiv:2409.19381.