Title: Rectified CFG++ for Flow Based Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Quality

Perceptual

Generation

T2I

Guidance

Sampling
Abstract: Classifier‑free guidance (CFG) is the workhorse for steering large diffusion models toward text‑conditioned targets, yet its naïve application to rectified flow (RF) based models provokes severe off–manifold drift, yielding visual artifacts, text misalignment, and brittle behaviour. We present Rectified-CFG++, an adaptive predictor–corrector guidance that couples the deterministic efficiency of rectified flows with a geometry‑aware conditioning rule. Each inference step first executes a conditional RF update that anchors the sample near the learned transport path, then applies a weighted conditional correction that interpolates between conditional and unconditional velocity fields. We prove that the resulting velocity field is marginally consistent and that its trajectories remain within a bounded tubular neighbourhood of the data manifold, ensuring stability across a wide range of guidance strengths. Extensive experiments on large‑scale text‑to‑image models (Flux, Stable Diffusion 3/3.5, Lumina) show that Rectified-CFG++ consistently outperforms standard CFG on benchmark datasets such as MS‑COCO, LAION‑Aesthetic, and T2I‑CompBench. Project page: https://rectified-cfgpp.github.io/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an improved Classifier-Free Guidance for Flows. The primary objective of paper is to enhance the quality of image generation for flow models. It provides both theoretical and practical evidence.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1, The method proposed in this paper is easy to understand and reproduce. Additionally, the authors provide the code.\n\n2, This paper improves upon state - of - the - art generative models and has significant frontier value.\n\n3, The paper provides abundant visual results, which are of high quality, effectively demonstrating the method's potential and reliability.\n\n### weaknesses\n\n1, Based on my understanding, the original process involves a timestep going through the neural network **twice**. \nThe proposed method changes this to a process where a timestep goes through the neural network **three times**.\nGiven this understanding,  the comparison experiment may not be fair.\n\n2,The primary contribution of this paper is to provide image details while suppressing artifacts. However, the paper lacks theoretical proof for artifact reduction. More details see Q5.\n\n3, The method consumes more time under identical conditions, which may restrict its practical application.\n\nQUESTIONS:\nQ1: As I understand it, the proposed method effectively converts the previous single-step process into a multi-step procedure. To elaborate, if the CFG runs for 20 timesteps, it would involve the model being invoked 40 times. In contrast, under the method presented in this paper, running for 20 timesteps would require the model to be invoked 60 times. Could the authors please confirm whether my calculation is accurate in this regard?\n\nQ2:  Whether the quantitative and qualitative experiments conducted in this paper were carried out under the same number of timesteps. If the calculation in Q1 is right, it seems that for a fair comparison, the CFG method should be evaluated with twice the number of timesteps. \n\nQ3: In Table 4, the evaluation is based on the number of sampling steps (NFEs). I am curious to know what the results would look like if the evaluation criterion were changed to the number of times the flow model is called instead. \n\nQ4: In line 5 of Algorithm 1, a random noise is added. Is there no coefficient needed for this random noise? Is this random noise enabled in this paper? Will the introduction of this randomness disrupt the manifold that the paper aims to pursue?\n\nQ5: Proposition 1 proves that there exists an upper bound, which ensures that the trajectory remains boundedly close to the target conditional flow path. This is fine. However, how does this ensure artifact reduction? Being close to the target conditional flow path doesn't seem to explain it. If I simply lower the CFG value, it will naturally be closer. Compared to directly adjusting the CFG value, what is the advantage of this paper?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Rectified-CFG++, a predictor-corrector based adaptive guidance method for diffusion models that combines rectified flows with manifold-aware conditioning. The authors claimed that it ensures stable and efficient sampling by anchoring updates near the data manifold and interpolating between conditional and unconditional velocity fields. Experimental results are provided to show that the method outperforms standard CFG across multiple large-scale models and benchmar\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\nAlthough the manifold constrained CFG has been proposed for diffusion models, it has not been exploited for rectified Flow model. Given the subtle and important differences between the rectified flow and diffusion, the scope of the paper is timely and could be potentially impactful.  Moreover, idea of using predictor and correct sampling, which is an important sampling scheme in flow and diffusion, for the CFG correction is also intriguing.\n\n**Weaknessess**\nAlthough the reviewer agrees that employing a predictor–corrector sampling framework is a promising direction for improving classifier-free guidance (CFG), the current manuscript does not convincingly validate this approach. In particular, while the authors emphasize theoretical contributions through Lemma 3.1 and Proposition 1, these results appear insufficient to support the claim that the proposed method performs manifold-constrained sampling. At best, they suggest that the sampling trajectories do not diverge excessively—but this does not imply that the trajectories accurately follow the data manifold. To rigorously demonstrate that the predictor–corrector scheme induces samples along the true data manifold, it would be more appropriate to analyze whether the resulting trajectories satisfy a continuity equation or the Kolmogorov Forward Equation (KFE) [1], which describe the evolution of probability densities under the stochastic process. Without such theoretical grounding, the core claim of geometry-aware sampling remains speculative. This lack of formal validation undermines the strength of the manuscript’s theoretical contributions and leaves the reviewer uncertain about the true efficacy of the proposed method.\n\n[1] Lipman, Yaron, et al. \"Flow matching guide and code.\" arXiv preprint arXiv:2412.06264 (2024).\n\nQUESTIONS:\nQ1: Validity of Theoretical Claims -- \nAs discussed above, the current theoretical analysis does not sufficiently support the claim that the proposed method ensures manifold-constrained sampling. Lemma 3.1 and Proposition 1 merely suggest bounded deviation of the sampling path, which is not equivalent to alignment with the true data manifold. To rigorously validate this claim, the authors are encouraged to provide a more formal analysis—e.g., by examining whether the proposed sampling scheme satisfies the continuity equation or the Kolmogorov Forward Equation (KFE). Such analysis would clarify whether the proposed predictor–corrector framework preserves the desired data distribution over time and substantiate its theoretical contribution.\n\nQ2: Ambiguity and Lack of Clarity in Figure 1 --\nThe current version of Figure 1 and its caption are ambiguous and uninformative. It is unclear what specific problem in existing methods the figure is intended to highlight, or how the proposed method addresses it. The visual does not clearly convey either the failure mode of standard CFG or the geometric benefit introduced by Rectified-CFG++. The authors should revise this figure to more clearly illustrate the motivation, contrast the behavior of existing vs. proposed approaches, and explicitly indicate what the readers should take away from it.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-25"}

comments: SUMMARY:
This paper proposes an adaptive predictor-corrector guidance method for rectified flow-based models, combining the deterministic nature of rectified flows with a geometry-aware conditioning rule. During inference, the model first follows the conditional RF prediction, then applies a correction step between the conditional and unconditional velocity fields.

STRENGTHS AND WEAKNESSES:
- Strengths
    - This paper is well written and easy to follow.
    - It introduces a novel, training-free methodology that improves efficiency.
    - The paper constrains the extrapolative nature of CFG, bringing it closer to the target trajectory.
- Weaknesses
    - The paper should include a comparison with other geometry-aware guidance methods, such as CFG++ (Chung et al., 2024), which also addresses off-manifold issues. A more detailed discussion of how the proposed method relates to or differs from CFG++ in mitigating these challenges would strengthen the paper.
    - The proposed method appears to be a variation of rectified flow that incorporates conditional updates. Since predictor-corrector (PC) schemes are already widely used in diffusion and flow-matching models, a clearer explanation is needed to distinguish this approach from existing PC-based sampling methods.
    - The method appears sensitive to the guidance scale hyperparameter. It would be helpful to compare its performance against CFG under the same guidance scale setting.
    - As the method is tailored to pre-trained flow-matching models, its generalizability and applicability to other model types remain questionable.

QUESTIONS:
Please refer to the weaknesses.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-27

comments: SUMMARY:
This paper aims to address the shortcomings of current CFG methods, particularly in the context of rectified flow models. Specifically, traditional CFG approaches tend to extrapolate, which often leads to the generation of artifacts. In contrast, the authors observe that rectified flows favor interpolation. Based on this insight, they design a method that keeps the sample on the manifold using a predictor-corrector integrator framework. The paper provides theoretical analysis, along with detailed ablations and experiments across a wide range of models and benchmarks.

STRENGTHS AND WEAKNESSES:
Strengths:

- The paper is compared with CFG on a wide range of latest strong models such as Lumina, SD3, SD3.5, and FluxDEV, therefore prove the soundness of the proposed method. 

- It's interesting to see that the proposed method can improve text rendering ability, which is usually a bottleneck in latest image generation models.

- The paper conduct evaluation on a wide range of commonly used benchmarks to support its claims, and the proposed method consistently outperforms or on par with the original CFG or other variants.

- The proposed algorithm doesn't incur significant computational cost compared with the original CFG. Therefore, it's easy to be used as a plug-and-play module during inference. 

Weakness:

- I didn't observe obvious weakness of this paper. It's well written and well supported by experiments. 

- In the tables, the author highlights almost all numbers, using colors like grey and orange, and also bolds the proposed method (Rect-CFG++). As a result, the presentation appears somewhat cluttered. It would be better to simplify the highlighting for improved clarity.

QUESTIONS:
NA

LIMITATIONS:
NA
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02