Title: Cross-fluctuation phase transitions reveal sampling dynamics in diffusion models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Models

Fluctuation theory

Phase Transitions

Sampling

Markov Chains

Interpretability.
Abstract: We analyse how the sampling dynamics of distributions evolve in score-based diffusion models using \emph{cross-fluctuations}, a centered-moment statistic from statistical physics. Specifically, we show that starting from an unbiased isotropic normal distribution, samples undergo sharp, discrete transitions, eventually forming distinct events of a desired distribution while progressively revealing finer structure. As this process is reversible, these transitions also occur in reverse, where intermediate states progressively merge, tracing a path back to the initial distribution.  We demonstrate that these transitions can be detected as discontinuities in $n^{\text{th}}$-order cross-fluctuations. For variance-preserving SDEs, we derive a closed-form for these cross-fluctuations that is efficiently computable for the reverse trajectory. %, thus tying cross-fluctuation dynamics to event formation within the desired distribution. 
We find that detecting these transitions directly boosts sampling efficiency, accelerates class-conditional and rare-class generation, and improves two zero-shot tasks--image classification and style transfer--without expensive grid search or retraining. We also show that this viewpoint unifies classical coupling and mixing from finite Markov chains with continuous dynamics while extending to stochastic SDEs and non Markovian samplers. Our framework therefore bridges discrete Markov chain theory, phase analysis, and modern generative modeling.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduced a study of score-based models using cross-fluctuations. The aim of the paper is to show that in the sampling process there are several transitions between distinct states. The paper illustrates that this states can be detected with the cross-fluctuation formalism and be used to speed-up the sampling and improve zero-shot use-cases.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is not really accessible to all the community. Even if the result is a purely mathematical one, we still have to remember that we are submitting the work to share the knowledge to most of the community. Perhaps, before diving into the mathematical formalism, I always recommend to give an highly level overview on the method and intuition behind the discovery. Due to the high-density of mathematical formalism without broader intuition, the paper is really hard to read. Half-way through the paper, I lost completely the scope of the paper. \n\nSaid so, I don't really understand what a \"transition\" is in this use-case. What does actually measure that there is a phase transition? Phase transition between what and what? In Physics, phase-transitions can be first or second order with their properties. Here, the use of transition between phase is used since the beginning but I haven't seen any definition on the states and/or transitions. In the algorithm, are you sampling with two score-based models simultaneously and measure the cross-fluctuations between the two? Is it the code available? \n\nI believe that this kind of approach is novel, I've never seen these concepts applied to score-based models. Experimental results show improvements in FID and inference time. However, I'd ask the authors to give a higher-level intuition in the main paper and part of the mathematical definition to the appendix. At the current state, I see a high-potential paper with lack in interpretability for most of the research community.\n\nQUESTIONS:\nSee Strengths and Weaknesses.\n\nLIMITATIONS:\nSee Strengths and Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper analyzes sampling dynamics in score-based diffusion models through cross-fluctuations, an nth-order central moment that measures when two sets of trajectories become indistinguishable under the forward noise process. It shows the connection between phase transitions with the trajectory  discontinuities of this statistic. For variance-preserving SDEs, the authors derive a closed-form expression that can be evaluated without additional simulations. These theoretical results lead to several methods that can detect merger points and safely skip the earliest reverse steps, yielding faster sampling and other guidance tricks without retraining.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper introduces a fresh mathematical viewpoint for studying diffusion models by bringing tools from fluctuation theory into the analysis.\n2. The study is thorough, moving from formal results to practical tests, and the theory-driven algorithm shows promising empirical gains.\n\nWeaknesses:\nMy main concern is presentation. To me, the key contribution is the new theoretical lens, while the experiments mostly serve as illustrations. The paper does not present clearly which theoretical results are formally proved and how to read them intuitively. It would help to make it explicit which parts are provable theory and which parts are theory-inspired heuristics.\n\nQUESTIONS:\n1. The Main Contributions section states that this work establishes \"a connection between ensemble based techniques in statistical mechanics as well as classical mixing-coupling results for Markov chains, using a generalized notion of phase transitions\". Could you provide a high-level explanation of this result, and comment on how novel or significant it is within the statistics literature?\n2. Typo: line 292 reads \"methodlologies\" but should be \"methodologies\".\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework for controlling sampling in diffusion models through the lens of fluctuation theory. The central insight is that during forward diffusion, different classes (data events) exhibit distinct fluctuation behaviors, which eventually converge (or \"merge\") as the process approaches pure noise. The authors define a normalized cross-fluctuation metric   to quantify distinguishability between events and use it to detect phase transitions — points at which classes become statistically indistinguishable. \nThey apply this framework to: accelerated sampling via early stopping, improved class-conditional guidance, enhanced rare-class generation, zero-shot classification using fluctuation signatures.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- originality: The use of fluctuation metrics as a lens to analyze diffusion dynamics is novel. Rather than relying on learned metrics or external classifiers, the method uses intrinsic model statistics.\n- applicability: The paper provides a practical algorithm (Algorithm 1). The method is evaluated across diverse settings: class-conditional sampling, rare class recovery, long-horizon compositional prompts, and even zero-shot classification.\n- significance: This approach has wide implications; among which acceleration of diffusion, improvement of rare-class generation, guiding conditioning more precisely, and reveal structure without training. \n- clarity (mostly): The paper is well-organized and mostly clear. That said, some parts, especially the explanation of how fluctuation metrics translate to guidance windows — require careful reading or visual aids.\n\nWeaknesses\n\n- clarity: The concept of fluctuations, merger times, and their link to class indistinguishability is dense and may be hard to parse without prior familiarity. The paper would benefit from intuitive visualizations or simplified examples early on.\n- Metrics/ interpretation of the numerical results for some applications: While precision/recall/coverage are well-adapted for generative models, the way numerical results illustrate the claims of the authors is not always clear. For instance, it is not clear how they indicate a greater rare-class coverage. the paper could be clearer about how this, especially since it doesn’t directly report class-wise sample counts.\n- Uncertainty in Merging Behavior: The notion that class k's merger time is based on indistinguishability from any other class (rather than all classes) is conservative but may underestimate useful guidance intervals. Discussion of this tradeoff is absent.\n\nQUESTIONS:\nSec 4.2: to what extent are the results sensitive to the threshold epsilon? It would have been interesting to give the results for various epsilons.\n\nSec 4.3: similar question: I understand that the criterion is whether the rare class has merged with any other class. Would the results be sensitive to a change of this criterion, e.g. the merging of the rare class with m classes with m>1? \n\nSec 4.4: would this method generalize to hierarchical or fine-grained classification settings (e.g., sub-breeds of dogs, rather than mnist classes)? How scalable is the covariance-based comparison in high-dimensional spaces?\n\nLIMITATIONS:\nThe authors did not really discuss the limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper presents a novel framework for understanding the sampling dynamics of diffusion models using cross-fluctuation, a concept from statistical physics. The authors propose a useful operator to capture the discrete transition during the sampling of diffusion models and demonstrate the effectiveness of the proposed method on five applications, including early stopped sampling, CFG sampling, rare-class generation, classification, and style transfer.

STRENGTHS AND WEAKNESSES:
Strengths:
- The paper is well-organized, and the application of the concept of cross-fluctuation on diffusion models is novel. 
- The proposed method provides a practical criterion, which can be computed with only the forward process, to capture the discrete phase transition in the sampling dynamics of diffusion models. 
- Such a criterion is shown to be practically useful in various scenarios. 

Weaknesses:
- The most crucial hyperparameter, the $\epsilon$ in Equation (3.1), requires more detailed illustration. The choice of $\epsilon$ determines the performance of the proposed method in all the included application scenarios. However, the criterion for choosing a proper $\epsilon$ seems to lack discussion, except for class-conditional sampling, where a criterion based on the largest eigenvalue is proposed. If a proper $\epsilon$ is sensitive to the dataset or task, it will still take a source-extensive sweep search to find the optimal one.

QUESTIONS:
I don't have further questions.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02