Generalization through variance: how noise shapes inductive biases in diffusion models

Authors: John Vastola
Primary area: generative models
Keywords: diffusion models, generalization, inductive biases, theory, infinite-width neural networks, generative models, path integral

Abstract:
How diffusion models generalize beyond their training set is not known, and is somewhat mysterious given two facts: the optimum of the denoising score matching (DSM) objective usually used to train diffusion models is the score function of the training distribution; and the networks usually used to learn the score function are expressive enough to learn this score to high accuracy. We claim that a certain feature of the DSM objective—the fact that its target is not the training distribution's score, but a noisy quantity only equal to it in expectation—strongly impacts whether and to what extent diffusion models generalize. In this paper, we develop a mathematical theory that partly explains this 'generalization through variance' phenomenon. Our theoretical analysis exploits a physics-inspired path integral approach to compute the distributions typically learned by a few paradigmatic under- and overparameterized diffusion models. We find that the distributions diffusion models effectively learn to sample from resemble their training distributions, but with `gaps' filled in, and that this inductive bias is due to the covariance structure of the noisy target used during training. We also characterize how this inductive bias interacts with feature-related inductive biases.

Decision: accept (poster)
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730698330390

thread_type:
official_review

<thread_object>
creator: reviewer_ayyq

<content>
summary:
The authors study the generalization in diffusion models. They highlight six key factors influencing generalization: noisy objective functions, forward process dynamics, nonlinear score dependencies, model capacity, architectural features, and the structure of the training set. They 
attribute the generalization of diffusion models to the fact that the training objective is in fact equal to the ground truth score function only in expectation. The key statement is that generalization occurs if and only if the V-kernel is not zero. Then they characterize the generalizability of diffusion models by analyzing the structures of V-kernel under several simplified conditions.

strengths:
1. Expressing PF-ODE in terms of the path integral is novel and offers important insight i.e, generalization occurs if and only if the V-kernel is nonzero.

2. The paper provides comprehensive study on three distinct cases through the V-kenrl: (i) memorization (ii) linear model (iii) NTK model. These results demonstrate V-kernel as a useful tool toward understanding generalization.

weaknesses:
(i) The authors claim that generalization occurs if and only if V-kernel is nonzero since otherwise the reverse sampling process can only produce training examples. However, this definition seems restricted since when diffusion models generalize, they not only differ from the PF-ODE dynamics, but generate "high quality" samples as well. In other words, the V-kernel should have certain benign properties. This point is not adequately addressed in the current paper.

(ii) In section 5 the authors derive the analytical form of the V-kernels under three circumstances. However, the resulting equations (9), (13) and (17) are really hard for me to interpret. Can the authors provide more explanation on the unique properties of these different V-kernels and how they shape the distributions corresponding to the reverse sampling process?

(iii) As a researcher focuses on empirical works, I don't find the results of this paper very useful. I think the paper could benefit from a further discussion on how the results can help improving practical diffusion models. 

(iv) Overall, despite the weaknesses I listed above, I think the results are interesting and worth being published.

questions:
See my questions in the weakness part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730721387665

thread_type:
official_review

<thread_object>
creator: reviewer_hd21

<content>
summary:
This paper investigates the generalization capabilities of diffusion models through the perspective of "generalization through variance," a novel concept where the inherent noise in the denoising score matching (DSM) objective used in training significantly influences model generalization. The authors develop a mathematical theory using a physics-inspired path integral approach, which reveals how the covariance structure of the noisy training target impacts inductive biases. The findings suggest that diffusion models are not merely reproducing training data but are capable of filling in 'gaps' that enhance their generative abilities.

strengths:
The paper's originality lies in its novel theoretical framework, which is a significant leap in understanding the mechanics behind diffusion models' ability to generalize. The quality of the theoretical analysis is high, and the writing is clear. The significance of the work is evident as it gives a new understanding of the generalization of diffusion models.

weaknesses:
1. The paper's main weakness is the lack of empirical validation of the theoretical claims. While the theoretical intuition is clear, the paper would be greatly strengthened by including simulations or experiments that demonstrate the practical impact of the "generalization through variance" phenomenon.
2. There is a lack of theory directly bridging the gap between variance and generalization error.

questions:
**1.** On the bottom of page 1 “At present, there is arguably no theory that describes…” It seems there exists a bunch of papers analyzing the generalization ability of diffusion models including their generalization time and its relation to data structure, see [1], [2].

[1] Li, P., Li, Z., Zhang, H., & Bian, J. (2023). On the generalization properties of diffusion models. Advances in Neural Information Processing Systems, 36, 2097-2127.

[2] Fu, S., Zhang, S., Wang, Y., Tian, X., & Tao, D. (2024). Towards Theoretical Understandings of Self-Consuming Generative Models. arXiv preprint arXiv:2402.11778.

**2.** It would be clearer to add more description to Figure 1, including the scale of the variance heat map, etc.

**3**. How does the "generalization through variance" compare with other known mechanisms of generalization in diffusion models, such as those related to model capacity and training set structure? In particular, is there a specific formula between V-kernel with any kind of generalization error (or its bound)? If so, the generalization could be more quantitively measured based on V-kernel. 

**4**. Could the authors provide empirical evidence or simulations that specifically illustrate the effects of the covariance structure on generalization as hypothesized in the paper?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730968586359

thread_type:
official_review

<thread_object>
creator: reviewer_doah

<content>
summary:
The main idea of the paper is that diffusion models generalize since the training step of estimating the score uses the “proxy score” as its target, which is a noisy version of the true score, or more accurately, a pointwise version of the score, conditioned also on the training data point. This introduction of this randomness into the nonlinear estimation process, and then into the diffusion process is claimed to lead to randomness in the generation process, and in turn, to generalization.

strengths:
Understating the empirical success of diffusion models in creating new samples that are similar to the training set is a timely and central question. The analysis made in the paper is dedicated to the generative question, and does not impose generalization measures from other problems (e.g., the MSE of the score function). The intricate details of the diffusion models considered are clearly discussed, and how they affect the generalization process. The paper is well written and conveys its ideas in an interesting way. 

An analytic stochastic path integral technique is used to derive closed-form expressions for the sampling distribution of new samples, with focus on the covariance structure, which highlights the role of the variance of the score proxy in the generation of new samples. It is also a direct performance measure compared to the MSE of the score. This technique is exemplified on two analyzable architectures.

weaknesses:
1) The paper assures that the generated samples are different from the training samples due to the noise in the score proxy, but this does not seem to explain why the generated samples are meaningful in some sense (e.g., have similar features to the training samples). If one assumes a ground truth distribution then this is obvious, but the authors emphasize that they refrain from that, but it is not clear what replaces this assumption (beyond the generated sampled being different from the training samples). 

2) Except for the appearance of a non zero covariance matrix in noise of the reverse diffusion process, the expressions in Propositions 5.1 and 5.2 are somewhat implicit. For example, it is not obvious how easy it is to compute them, or how its form affects generalization.

questions:
1. Line 67: What is exactly meant by “boundary regions” ? One example is regions that are close to multiple training points, but how these are defined in general ? As high likelihood regions ? This term is used repeatedly afterwards so it would be good to accurately define it. 

2. Line 75: How does the claim “models generalize better when they are somewhat underparameterized” agree with the (Karras et al, 2024) paper mentioned on line 48 ? 

3. Line 78: Why interpolation considered a non-trivial generalization ? 

4. Line 138: It would be better to introduce the proxy score before (4). 

5. How is the operator \mathcal{D} is defined on (6) ? 

6. Section 4: It is explained that even using the generated samples to estimate the score leads generalization. But what is the starting point of the process X_T (the noise point). Isn't this point generated from the training data in a noisy manner, and this also contributes to generalization ? 

7. Line 207: Once the higher order terms in (7) are neglected, it is mentioned that this implies that the estimator distribution is approximately Gaussian. However, due to the Gaussianity of the forward process, doesn't this tacitly also approximate the data distribution as Gaussian ? In other words, if we assume that the data distribution is Gaussian, would the result be the approximated integral of (7) ?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732754451850

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global reviewer response

comment:
We thank all of the reviewers for their comments, which we feel have substantially improved and clarified the paper. We are still working on implementing the associated changes, so we will not be able to provide a modified PDF by the deadline, but the major and minor changes are as follows.

**Major changes.**

**New main text section.** We are adding a section titled "**Generalization through variance: consequences and examples**" after the main theoretical results and before the discussion. The goal of this section is to both empirically validate our mathematical results, and to (in the context of various simple experiments) show how generalization through variance affects distributions in practice. To ease both interpretability and computation, we are conducting these experiments in simplified 1D and 2D settings. 

The specific layout of the section is as follows. First, we discuss various "benign" properties of generalization through variance (e.g., variance is not added far from training examples, and does not change the dimensionality of the data manifold) that suggest that it produces a 'reasonable' type of generalization. This is majorly due to a suggestion by reviewer AyYq. 

Next, we conduct small experiments to show how generalization through variance relates to filling (or exaggerating) gaps between training examples, the existence of outliers and duplicates in training data, asymmetry in the forward process, and different feature sets (in the case of the linear score estimator). As previously mentioned, we consider only 1D and 2D point cloud data distributions here.

Finally, we comment on the relationship between the form of the V-kernel and generalization error, as suggested by reviewer hd21. This turns out to be a difficult problem to address mathematically in general, so we do this in a few special cases.

**Minor changes.**

- **New appendix on benign properties of generalization through variance.** This appendix provides more mathematical detail regarding why one generically expects generalization through variance to be 'reasonable' given its relationship to the proxy score covariance. This is due to a suggestion by reviewer AyYq.
- **New appendix on boundary regions, and how they relate to the proxy score covariance.** Because we use the concept of 'boundary regions' throughout the paper, we think it would be helpful to clarify what we mean. We discuss what we mean mathematically and relate this notion to concepts like "score blindness" which have been previously discussed in the diffusion model literature. This is due to a suggestion by reviewer DoAH.
- **New appendix on generalization error.** This appendix computes generalization error (which we formulate in terms of the KL-divergence between the typical learned distribution $[q]$ and some ground truth distribution $p_*$) in a few special cases, and in particular shows how it depends on the form of the V-kernel. The special cases of interest essentially correspond to when the reverse process noise is very large, and when it is very small. This is due to a suggestion by reviewer hd21.
- **New appendix presenting main results in terms of alternative formulations.** While we chose to present our main results in terms of score-matching, one can also derive similar results for other formulations of diffusion modeling. We present our main results (the three propositions from Sec. 4-5) for two other formulations used in practice: the 'denoiser' formulation (used by, e.g., Karras et al. 2022), and the 'noise prediction' formulation (used by, e.g., the original Stable Diffusion paper). This is mostly just a quality of life change to make our results more accessible to other researchers.
- **Clarification of various technical points.** We add additional commentary regarding certain technical issues, e.g., the relationship between assuming the score estimator distribution is Gaussian and assuming the data distribution is Gaussian (reviewer DoAH), and what kind of generalization counts as 'nontrivial' in this setting (also reviewer DoAH).
- **Small quality-of-life improvements.** We fix various small issues pointed out by reviewers, e.g., the introduction of the proxy score (reviewer DoAH), and adding more description to the Figure 1 caption (reviewer hd21).
- **Additional citations.** We have cited additional related work, including the two papers suggested by reviewer hd21.

We thank all reviewers for their patience as we work on implementing these changes and additions.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper studies how diffusion models generalize when they are trained using denoising score matching. The analysis in the paper is intriguing and focuses on how the score is targeted with a noisy version (the conditional score) with regression rather than directly impacts the generalization. They then develop theory to understand this kind of generalization through variance. The paper is clearly written. There were weaknesses written by the reviewers around empirical validation and some of the assumptions, but the concerns were minor enough to have all the reviewers accept this paper.