Title: DISC: Dynamic Decomposition Improves LLM Inference Scaling
Venue: NeurIPS 2025
Accepted: yes
Keywords: Inference scaling

search

decomposition

LLM
Abstract: Inference scaling methods for LLMs often rely on decomposing problems into steps (or groups of tokens), followed by sampling and selecting the best next steps. However, these steps and their sizes are often predetermined or manually designed based on domain knowledge. We propose dynamic decomposition, a method that adaptively and automatically partitions solution and reasoning traces into manageable steps during inference. By more effectively allocating compute -- particularly through subdividing challenging steps and prioritizing their sampling -- dynamic decomposition significantly improves inference efficiency. Experiments on benchmarks such as APPS, MATH, and LiveCodeBench demonstrate that dynamic decomposition outperforms static approaches, including token-level, sentence-level, and single-step decompositions, reducing the pass@10 error rate by 5.0%, 6.7%, and 10.5% respectively. These findings highlight the potential of dynamic decomposition to improve a wide range of inference scaling techniques.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel test-time inference strategy called DISC, which adaptively decomposes the generation process of large language models (LLMs) by dynamically adjusting the step granularity. Instead of relying on static token-level or sentence-level generation, DISC estimates the “difficulty” of each step and decides whether to accept or further expand it using a z-score-based criterion. The method is model-agnostic, requires no additional training, and is compatible with various search paradigms such as greedy decoding, beam search, and MCTS. Experiments across multiple reasoning benchmarks (APPS, MATH, LiveCodeBench) demonstrate notable performance gains under constrained compute budgets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is motivated by a practical problem and is clearly written, making it easy for readers to follow.\nDISC is easy to plug into existing inference pipelines, and doesn’t require training or model changes.\n2. Solid evaluation across tasks and models, and good ablations around its design choices (e.g., temperature, thresholds).\n\nWeaknesses:\n1. A number of recent papers (e.g., DynScaling [1], RINS [2]) also explore ways to adapt compute usage during inference. They differ in method—e.g., using multi-armed bandits, recursive refinement, or heuristic difficulty estimates—but share a similar motivation. The paper doesn’t really discuss these. It would help to clearly highlight what DISC does differently (especially around the level of granularity or how compute decisions are made), and what the practical gains are. \n2. The current z-score–based decomposition decision appears to be heuristic. Is there any theoretical justification or learned variant that could better adapt to different task domains or model behaviors?\n3. While DISC is compatible with search methods like beam or MCTS, the interaction mechanisms are not well explained. Does DISC affect search efficiency, or introduce redundancy in rollouts?\n[1] Wang F, Wan X, Sun R, et al. DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Sampling[J]. arXiv preprint arXiv:2506.16043, 2025.\n[2] Alabdulmohsin I, Zhai X. Harnessing Language's Fractal Geometry with Recursive Inference Scaling[J]. arXiv preprint arXiv:2502.07503, 2025.\n\nQUESTIONS:\nPlease refer to Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel framework for dynamic decomposition in modern reasoning or code generation LLMs, named DISC. DISC extends search strategies like MTCS and beam search by adaptively partitioning solution steps to put more computational resources on harder parts, thus improving effectiveness of inference. The authors performed experiments on APPS, MATH, and LiveCodeBench benchmarks against several static baselines, including token-level, sentence-level, and single-step decomposition. achieving up to 10% reduction in error relative to the baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* The paper rests on a simple yet effective idea to change the problem decomposition steps based on the insight that prefix statistics (which correlate with the promising reward improvement) can aid the split, leading to larger or smaller steps. The paper also shows that the DISC technique integrates with the state-of-the art LLMs, including open source and commercial ones. \n\n* Evaluation shows reasonable improvements over various baselines, given pass@k and pass@token metrics. Overall, I found the results in this section convincing, suggesting that DISC's adaptive partitioning improves correctness with the same token/sample budget over the alternative simple baselines. While the improvements are not enormous, they are significant enough to merit publication of this work and the overhead of the technique is low (assuming an appropriate reward model exists). \n\n* Several ablation studies show the impact of several parameters on the quality of the search, strengthening the main results. The authors further support their arguments with an extensive appendix with additional experimental studies and mathematical derivations.\n\n\nWeaknesses:\n\n* The paper does not explicitly discuss the limitations of the proposed approach. For instance, the presentation of how self-generated test validation proceeds is currently terse and the details should be clarified both in the evaluation section and discussed as a limitation. \n\n* (minor) Algorithm: In Algorithm 1, Y_b is a set whose elements are pairs of elements. However the next line is applying the functions max and argmax (presumably on 2nd and 1st element, respectively). Update the presentation so that the computation is mathematically well-defined. \n\n* (minor) Formatting: try to keep the colors consistent. For instance, Fig 7 has different colors for DISC across different benchmarks. Figure 9 would be more understandable if it had means with standard deviations as bars. Also, the figure caption should explain the dotted lines on that plot.\n\nQUESTIONS:\n- Figure 6a present open-source models. It is unclear which version of the model is base (if any) and which one is DISC (if any). Please clarify.\n\nLIMITATIONS:\nThe paper should contain a clearly marked list of limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses a timely and important challenge in large language model (LLM) inference: how to dynamically allocate compute by adaptively decomposing reasoning steps during decoding. The proposed DISC method introduces a dynamic step-size adjustment algorithm guided by statistical reward estimations (e.g., z-scores).\n\nSTRENGTHS AND WEAKNESSES:\n1. The paper proposes a seemingly effective decomposition mechanism that aims to reduce redundant sampling and better allocate compute across reasoning steps.\n2. The experimental evaluation is relatively thorough.\n\n\n## Weakness：\n1.\tAlthough the paper claims DISC performs dynamic step selection based on reward distribution statistics, it lacks a principled theoretical justification for this dynamic behavior. It is unclear what precisely governs the dynamic adaptation—apart from the z-score of a small number of sampled candidates.\n\n2.The method depends strongly on hyperparameters such as the initial partition ratio, the reward normalization scheme, and update rules. Therefore, this dynamic decomposition is fundamentally similar to simply pre-setting a static decomposition strategy.\n\n3.A key premise of the paper is to dynamically allocate compute based on the difficulty of different steps in the decomposition. However, the notion of step difficulty is never formally defined, measured, or validated. There is no metric to demonstrate that the method is indeed allocating more compute to harder reasoning steps.\n\n4.The method is claimed to improve inference efficiency, however, no concrete analysis of runtime is provided. Figure 5 reports only task accuracy metrics such as Pass@token or Pass@k, which do not isolate computational cost. While the authors vaguely claim negligible runtime overhead (Sec. 4.1 and Appendix Fig. 40), no detailed comparison of actual time or resource usage is shown. This significantly weakens the credibility of the efficiency claim.\n\n5.\tThe method is only compared against a few relatively simple and static decomposition schemes. The paper omits recent and stronger baselines from the literature on adaptive inference and multi-step reasoning with LLMs.\n\nQUESTIONS:\n1.\tInclude explicit compute-efficiency metrics\n2.\tProvide more concrete visualizations or case studies showing how DISC reallocates compute dynamically in real examples.\n3.\tMore questions can be seen in the Weakness part.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper introduces DISC, a dynamic decomposition approach to inference in large language models that adaptively introduces partitions among the solution steps to reflect the estimated difficulty of the solution, instead of using a fixed and hand-designed set of solution steps or uniform steps. The recursive reward-based process used by DISC decides to dedicate more compute to difficult reasoning steps and less to easy ones and fits naturally with standard search algorithms (greedy, beam, MCTS). On various benchmarks of code generation and mathematical reasoning (APPS, MATH, LiveCodeBench), DISC achieves on average large improvements in inference efficiency and correctness compared to static decomposition baseline results.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper is well-motivated and tackles a meaningful problem in LLM reasoning
2. Extensive experiments on multiple benchmarks with strong improvements 
3. Ablation studies are thorough 

Weaknesses:
1. The theoretical results are built on strong assumptions that may not necessarily hold in practice. Moreover, insights that DISC will eventually find the optimal solution as the search budget increases are fairly straightforward and hold for any flexible search algorithm. 
2. The z-score acceptance criterion needs further justification
3. The paper lacks analyses of any failure modes or empirical limitations of DISC

QUESTIONS:
Could the authors clarify the 3 points raised in the Weaknesses section?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03