reviews: 
Comments: The authors study the issue of generalization and memorization in diffusion
models. Diffusion models have as a backbone trained denoisers, and it has been
observed that the use of powerful deep networks as denoisers can lead to
situations where diffusion models memorize their training data (rather than
being able to generate novel images from the high-dimensional data
distribution), whereas in other cases the models seem to generalize. The authors
study this for a specific class of deep network denoisers (bias-free CNNs) both
empirically and theoretically. Empirically, they show that training these
denoisers on CelebA and LSUN bedrooms for varying training set sizes witnesses a
clear transition (qualitative and quantitative) between "memorization"
performance of the trained diffusion model when the training set size is small,
and "generalization" performance when it is sufficiently large. The authors
hypothesize that the ability to generalize with relatively few samples from the
high-dimensional distribution is due to inductive biases in the deep network
denoiser, and in particular they make a connection with classical ideas on
denoising from harmonic analysis to posit that DNN denoisers are biased towards
"geometry-adaptive harmonic bases" (eg bandlets/wedgelets).  They test this
hypothesis empirically, demonstrating that these trained denoisers learn
bandlet-like bases in which to perform shrinkage on toy image classes where such
bases are optimal (horizon classes), and moreover that they persist in learning
these types of bases even for classes of signals where they are suboptimal
(image articulation manifolds).
Strengths: - The paper presents detailed (but not overly technical/complex) background on
  diffusion models, to allow a broad audience to appreciate the experimental
  results. (I do wonder here if it might be helpful to present somewhere the
  functional form of the 'optimal' denoiser for the empirical risk, eg as it's
  done in Karras et al 2021, to ground the memorization issue a bit more -- I
  find this concept helpful to have in mind as I'm reading the background of the
  paper.)

- The core question the paper considers -- issues of generalization in diffusion
  models trained to generate samples from high-dimensional data distributions --
  is both important and currently without a completely satisfying
  conceptual/mathematical explanation, despite much concurrent work. The authors
  present a compelling empirical study of this issue under controlled
  conditions, showing (among other things) that it is indeed real, and give a
  plausible theoretical explanation for why it occurs.

- The theory of inductive bias that the authors put forth is nontrivial,
  involving, through equation (6), a specific representational formula for
  piecewise-linear denoisers, and thereby a connection to classical shrinkage
  estimators.
Weaknesses: - It seems that the theoretical framework
  posited in section 3 is specific to the particular denoiser architectures
  being studied in the paper (BF-CNN). For example, PixelNet++-type denoisers
  used in modern diffusion models do not seem to be amenable to a decomposition
  like equation (6), because they involve attention layers and positional
  embeddings (presumably with affine components) to implement different
  conditioning operations. It would be good if the authors could comment on this
  issue, and how they see the theory extending to this modern setting.
Recommendation: 8: accept, good paper
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 4 excellent
Clarity: 4 excellent