Title: Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: training-free guidance

inference-time scaling

diffusion models

flow models

tree search
Abstract: Training-free guidance enables controlled generation in diffusion and flow models, but most methods rely on gradients and assume differentiable objectives. This work focuses on training-free guidance addressing challenges from non-differentiable objectives and discrete data distributions. We propose TreeG: Tree Search-Based Path Steering Guidance, applicable to both continuous and discrete settings in diffusion and flow models. TreeG offers a unified framework for training-free guidance by proposing, evaluating, and selecting candidates at each step, enhanced with tree search over active paths and parallel exploration.  We comprehensively investigate the design space of TreeG over the candidate proposal module and the evaluation function, instantiating TreeG into three novel algorithms. Our experiments show that TreeG consistently outperforms top guidance baselines in symbolic music generation, small molecule design, and enhancer DNA design with improvements of 29.01%, 26.38%, and 18.43%. Additionally, we identify an inference-time scaling law showing TreeG's scalability in inference-time computation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a family of methods for inference-time control of diffusion generative model with nondifferentiable rewards. The method involves keeping a set of samples and filtering or resampling the set of proposals generated from those samples at each diffusion step. The method is evaluated on a wide array of discrete generation tasks, including symbolic music, molecules, and DNA. Scaling experiments are performed to establish the impact of key hyperparameters.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\nThe paper's key conceptual idea, that of maintaining a set of samples that is steered and resampled during inference via a reward function, is well worth articulating and presenting to the community. It mirrors important developments in continuous diffusion steering (Feynman-Kac steering) that demonstrates the benefits of maintaining a set of samples, a la SMC, rather than steering individual trajectories. The coverage and diversity of the experiments is noteworthy and shows promising results.\n\n**Concerns**\nWhile the core message and idea of the submission are sound and potentially impactful, there are concerns about the manner of presentation and execution of the method.\n* Despite the appealing intuition of maintaining an active set, the main experiments are nonetheless conducted with $A=1$. With this departure, the exposition in the abstract, exposition, and figures are misleading.\n* The theorems in the continuous case suppose that the data are Gaussian and the reward function linear. These restrictions are so severe as to perhaps disqualify the theorem from being a main result. Certainly, it cannot be said in exposition that \"TreeG-SC yields a distribution T that close approximates...\" without qualification.\n* There is not very much exploration about how to define the intermediate value functions, which seems critical to the performance of the method. Evaluating the terminal reward at samples x_1 | x_t obtained in one step might not work, as those samples could be far out of distribution of x_1.\n* There appears to be a somewhat exaggerated difference between TreeG-SC and TreeG-SD, differing only in whether the variance of x_1 | x_t is accounted for or whether we use the conditional mean. In the discrete case, they appear identical.\n* With ranking based selection, the procedure reduces to a intuitive heuristic procedure. Unlike SVDD, we do not know what distribution the procedure will sample. This makes it difficult to reason and fairly compare the performance with methods that do sample a known distribution - the procedure could be optimizing the reward but resulting in diversity collapse or missing modes in hard to reach local minima.\n* It seems that whenever the results in a column are not favorable to the authors, the results in that column are not bolded (Tables 1, 2, 3).\n* The paper uses \"scaling law\" to refer to any monotonic decrease in loss - this is not accurate.\n\nAltogether, the core idea of the paper is strong, but the experiments do not focus on all of the most important aspects of the idea and overemphasizes points that are not really the paper's strengths (i.e., theoretical bounds). I do not anticipate that the authors will have time to dramatically reorganize the presentation of results during rebuttal; however, I am open to raising the score if the authors can argue why the procedure. as implemented in the main experiments, corresponds to what is claimed in the abstract and introduction.\n\nQUESTIONS:\nNo specific questions for authors.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors proposed tree search methods for inference time (black-box) guidance for both diffusion models and flow models(discrete cases).  Two schemes are proposed for tree branch expansion and value function estimation.  The authors further provide theoretical bounds of the difference between the algorithm-induced distribution $\\hat{\\mathcal{T}}$ and target conditional transition distribution $\\mathcal{T}(\\cdot | \\boldsymbol{x}_t,y )$ for both schemes.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. This paper provides tree search methods for inference time (black-box) guidance for both diffusion models and flow models(discrete cases).  Although [Huang et al. 2024] proposed a method that selects the best one at each time t for inference time  guidance, this paper generalizes it well for both diffusion models and flow models.  \n\n2. This paper provides theoretical error bounds of the difference between the algorithm-induced distribution $\\hat{\\mathcal{T}}$ and target conditional transition distribution $\\mathcal{T}(\\cdot | \\boldsymbol{x}_t,y )$ for both cases. It directly reveals the relationship between the approximation error and the tree Branch-Out size $K$. \n\n\n**Weaknesses**\n\n1. Theorem 2 for the discrete case is a bit weird.  It seems that given the same Branch-Out size $K$, the high-dimensional case can achieve a much smaller error $\\epsilon$ compared with the low-dimensional case. Or equivalently, to achieve a fixed error $\\epsilon$, the high-dimensional case requires a smaller  Branch-Out size $K$ compared with the low-dimensional case. \n\n2. The assumptions of the error bound for the continuous case are too strong. The requirement of the data follows a Gaussian distribution, and the objective function is linear is too restrictive. If we already know this nice form, we do not need to take diffusion models. We can easily start from the Gaussian; other techniques may get better results.  \n\n3. In the black-box guidance area, usually, the main challenge is the query efficiency. For example, in drug discovery tasks, the number of wet lab experiments or high-computation-cost docking evaluations is the bottleneck. In targeted image generation tasks, the number of collections of users' preferences or simply LLM API calls for scoring is the bottleneck. However, no comparison against baselines regarding query efficiency is evaluated in the current draft, which is not convincing enough. \n\n4.  It seems that all the evaluations in the experiments are on simple tasks.  In the drug discovery area, the Vina Docking score is usually more challenging to optimize compared with QED and SA. The QED and SA are easily led to overfitting. However, evaluations regarding the Vina Docking score are missing.  In addition,  no comparison is performed on the image domain. The image target generation tasks are high-dimensional, which makes them more challenging for the value function estimation. \n\n5. Part of the math or symbols are not clear enough.\n\n- The $R_t$, $R_t^{(d)}$ and $R_t( x_t, j | x_1)$ for the flow models are not well explained.  Is it a typo for the symbol on Line 117?\n\n- The distribution $\\hat{\\mathcal{T}}$  depends on the time t, but the error bound in the Theorem does not show this explicitly. \n\n6. Some prior related works on the inference-time black-box guidance for diffusion models are not discussed.  I list a few of them below. \n\n[1] Tang et al. Tuning-free alignment of diffusion models with direct noise optimization. arXiv:2405.18881, 2024.\n\n[2] Tan et al. Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generation. ICLR 2025.\n\n[3] Lyu et al. Covariance-adaptive sequential black-box optimization for diffusion targeted generation. arXiv:2406.00812, 2024.\n\nQUESTIONS:\nQ1.  Please provide more discussion and explanation about Theorem 2 for the discrete case and the assumptions of the error bound for the continuous case.\n\nQ2. If possible, please provide comparisons against baselines regarding query efficiency.\n\nQ3. If possible, please provide evaluations on the Vina Docking score and image generation tasks.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a training-free diffusion guidance algorithm applicable to general objectives with non-differentiable rewards and discrete data distribution. The proposed algorithm (TreeG) is based on a tree search along the diffusion sampling trajectory. The authors first state a generic tree-search framework (Section 4), and instantiate TreeG with different design choices (Section 5). The TreeG algorithm is proved to approximate the true posterior distribution closely with resampling. Experimental results on generating symbolic music, small molecules and enhance DNAs show clear advantage of TreeG against existing guidance methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- This paper provides a generic search-based framework for guiding score-based models for general objectives and using both continuous and discrete diffusion models.\n- The discussion that connects existing methods to TreeG by showing them as special cases of TreeG is inspiring.\n- The performance of the TreeG algorithm is rigorously guaranteed.\n- Experimental results show advantange of TreeG compared to existing methods. The experiments encompass different tasks in different domains. The scaling ability of TreeG also looks promising.\n\nWeakness:\n\n- Despite being training-free, the computational cost of the tree search-based algorithm seems to be high. A comparison or discussion on computational efficiency of this inference-time method compared to fine-tuning methods (e.g., RTB[1]) is missing.\n- While TreeG generate samples with better targeted objectives in the experiments, the distributional similarity is sometimes worse (e.g., when increasing the guidance strength in DNA design tasks). It is unclear whether the generated samples satisfy the conditional distribution $p(x|y)$ by only looking at the reported metrics. \n\n[1] Venkatraman et al. Amortizing intractable inference in diffusion models for vision, language, and control. NeurIPS 2024.\n\nQUESTIONS:\n- Could the authors also comment on the scalability of TreeG in terms of the dimensionality of the data distribution? Also, is there any theoretical insight on how tree search-based algorithms perform when the objective/reward functions are sparse in the search space, i.e., when most of the samples have low rewards?\n\nLIMITATIONS:\nYes, the authors address limitations in Appendix B.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
TreeG reframes training-free guidance as a tree-search over inference paths, enabling diffusion and flow models to optimize non-differentiable or discrete objectives without extra training. Three instantiations( TreeG-SC, TreeG-SD, and TreeG-G) are theoretically supported and outperform prior guidance methods on symbolic music, molecule, and DNA generation, while exhibiting a clear compute-versus-performance scaling law. In summary TreeG is a well-motivated step toward gradient-free control of diffusion and flow models, offering both provable guarantees and tangible empirical gains. Broader domain validation, ablations, and alignment between theory and implementation would be beneficial before claiming universal guidance capability.

STRENGTHS AND WEAKNESSES:
Strengths

* Consistent empirical wins across three distinct domains of application.

* Sensible supporting empirical analysis.

* Positive practical implications across widely relevant application domains

Weaknesses

* Domain coverage: three synthetic/benchmark tasks. Lack of an argument towards the selection criteria for the target tasks.

QUESTIONS:
Could the authors elaborate on the:

* Selection criteria for the target tasks? What is the argument for universality over this specific set?

* Theoretical assumptions (Gaussian data, linear objectives) may limit generality. Can you make more explicit the assumptions in which the model should perform?

* Possible reproducibility and transferability gap across tasks due to many hyper-parameters (e.g., Monte-Carlo N, temperature α) tuned per task without ablation. Can you comment on this?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-05