Title: Towards Understanding the Mechanisms of Classifier-Free Guidance
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion model

classifier free guidance

mechanism interpretation
Abstract: Classifier-free guidance (CFG) is a core technique powering state-of-the-art image generation systems, yet its underlying mechanisms remain poorly understood. In this work, we first analyze CFG in a simplified linear diffusion model, where we show its behavior closely resembles that observed in the nonlinear case. Our analysis reveals that linear CFG improves generation quality via three distinct components: (i) a mean-shift term that approximately steers samples in the direction of class means, (ii) a positive Contrastive Principal Components (CPC) term that amplifies class-specific features, and (iii) a negative CPC term that suppresses generic features prevalent in unconditional data. We then verify these insights in real-world, nonlinear diffusion models: over a broad range of noise levels, linear CFG resembles the behavior of its nonlinear counterpart. Although the two eventually diverge at low noise levels, we discuss how the insights from the linear analysis still shed light on the CFG's mechanism within the nonlinear regime.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the phenomenon of classifier-free guidance via the lens of linear diffusion models. The authors first show empirically that a linear diffusion model can approximate the results of diffusion models, then start the analysis from the linear diffusion model. The main proposition from the authors is that the CFG enables the sampling process towards the following three terms: (1) mean-shift towards the class mean; (2) a positive Contrastive Principal Components (CPC) term amplifying the class-specific features; and (3) a negative CPC term that suppresses generic features prevalent in unconditional data. Finally, the authors also discuss the extension to non-linear cases.\n\nSTRENGTHS AND WEAKNESSES:\nI have worked on some practical aspects of diffusion models, but not the theoretical side. I tried my best to follow the maths and compared them with my observations in actual experiments. Based on these, I have the following comments.\n\n# Strengths\n\n- It is intriguing to see the connection between diffusion models and a simplified linear diffusion formulation (Fig. 1).\n- The observation and statistics of class specifity is reasonable and make the paper well-motivated (Sec. 3.1).\n- The authors proposed the three terms explaining the intra-class and inter-class relationships, which aligns well with the intuition of how CFG assists generation.\n- The experiments conducted by the authors, ruling out the assumptions and factors gradually, further reveal the insights of understanding the effects of the three terms.\n\n# Weaknesses\n\nI am not working on the theoretical side of the diffusion models, so my question might not apply to the theory researchers. Specifically, I have reserved opinions toward the Equation 12, the ODE formula splitting into the three terms proposed by the authors.\n\nMy concern is that: under the assumptions of the linear model and Gaussian distribution, it seems intuitive that a CFG formula SHOULD correspond to the three terms: $p(x|c)$ --> mean shift, $p(x|c) - p(x| \\phi)$ the components between class-specific features and unconditional features. It is true that calculating the weights of the terms is challenging, but I am curious about how do the theory researchers judge that a conclusion is non-trivial?\n\nQUESTIONS:\nSee weaknesses above\n\nLIMITATIONS:\nYes, the authors have discussed limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides an in-depth analysis of Classifier-Free Guidance (CFG) in diffusion models, uncovering a bias caused by mean shift when the zero-mean assumption in data samples is violated. It proposes tuning the CFG coefficient to compensate for this bias. Empirical experiments support these findings, demonstrating the mean-shift phenomenon and offering a simple, practical adjustment to correct it.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nThe analysis of CFG is thorough. The authors first demonstrate that naive conditional sampling tends to yield low-quality samples that lack distinct class-specific features, which they attribute to the overlap in class covariance matrices.\n\n\n## Weakness\n\n* The setup for linear Gaussian diffusion is not clearly defined—how well does it approximate real-world diffusion processes?\n\n* The rationale for applying Contrastive Principal Component Analysis (CPCA) is unclear; further justification for its suitability would strengthen the analysis.\n\n* The study evaluates only EDM as the diffusion model. It's uncertain whether the observed effects generalize to other types of diffusion models.\n\n* The paper lacks actionable insights or design implications for improving existing Classifier-Free Guidance (CFG) based on its findings.\n\n* There is insufficient quantitative evaluation of how varying CFG configurations affect generation quality.\n\nQUESTIONS:\n## Questions:\n* In line 171, as discussed in Section 2.1: For high to moderate noise levels, deep diffusion models can be well approximated by linear models, particularly when network capacity is limited and training is insufficient. Could you provide both an intuitive and a quantitative justification for why the linear approximation is considered sufficiently accurate in this regime?\n\n* In line 178, it is stated that \"CFG significantly increases the inter-class separation.\" Given that FID is computed from the overall sample mean and covariance, how does increased inter-class separation affect the overall FID score? Could this separation inadvertently inflate FID despite visual improvements?\n\n* Based on the insights presented in this paper, do you envision any new directions or improvements to the design of existing CFG methods?\n\nLIMITATIONS:\n## Limitations:\n\n* The study evaluates only EDM as the diffusion model; it remains unclear whether the observed effects generalize to other diffusion architectures.\n\n* The paper does not offer actionable insights or design improvements for existing Classifier-Free Guidance (CFG) methods based on its analysis.\n\n* There is a lack of quantitative evaluation assessing how different CFG configurations influence generation quality.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to understand classifier-free guidance (CFG) from a theoretical perspective. To simplify the analysis, the authors first study a linear model and then explore how the insights might extend to real-world diffusion models. They find that linear CFG improves generation quality through three key components: (i) a mean-shift term that roughly guides samples toward the class means, (ii) a positive contrastive principal components (CPC) term that enhances class-specific features, and (iii) a negative CPC term that suppresses generic features commonly present in unconditional data. The experiments are conducted on simple linear models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Analyzing CFG theoretically to understand its mechanism is interesting and motivating.\n\n2. The theoretical formulation would be helpful to propose new guidance strategies.\n\nWeaknesses:\n\n1. While I appreciate the analysis in linear models, the extension to non-linear regime and real-world diffusion models seems weak. E.g., the argument in L349: “empirically ..” not not convincing, since I don’t see much improvement in the visual results in Fig. 7, with PCs, or CPC. Guiding with PCs even decrease the quality compared to CFG, why that's the case? \n\n2. In Fig.7, sample 3, guiding with CPC, PC, change the color of the car, can you explain this? Do PCs capture different visual attributes? But in that case, how the prompt alignment is performed, it's not clear to me.\n\n3. The qualitative results are not convincing. E.g., Fig. 5, I see non-linear CFG, performs better than CPCs. It can be useful to understand CFG, but how it's helping to improve CFG in terms of generation quality and prompt fidelity remains unclear.\n\n3. CFG is an inference time strategy, not involving any training, then how class statistics are affected? Please clarify.\n\n4. Experiments need to be done on recent diffusion models, with higher quality and resolutions to actually see any meaningful improvement.\n\n5. Can this analysis be extended or applied to classifier guidance?\n\n6. In L294, whats FD _{DINOv2} ?\n\n7. The flow and format of the paper needs to be improved. The figures should be improved overall, both in quality and formatting (labels etc).\n\nQUESTIONS:\nPlease address the weakess.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}