reviews: 
Comments: Diffusion generative models that use denoising DNNs have surpassed previous methods of learning probability models from images. The authors introduce a methodology to evaluate the properties of the trained denoiser and the density from which the data are drawn. They show empirically that diffusion models can converge to a unique continuous density model that is independent of the specific training samples. The convergence exhibits a phase transition between memorization and generalization as training data grows. They demonstrate that two denoising DNNs trained on non-overlapping subsets of a dataset learn nearly the same score function and, thus, the same density, indicating the existence of powerful inductive biases in the DNN architecture and/or training algorithm. The inductive bias of the network appears through the best basis, which is a geometry-adaptive harmonic basis (GAHB) when trained on photographic images. The DNN denoisers achieve near-optimal performance for the Cα class of images. They also investigate the inductive biases that enable this rapid convergence and show that DNN denoisers perform poorly for distributions whose optimal bases are not GAHB. For images drawn from low-dimensional manifolds, DNN denoisers achieve an accurate basis for the subspace and incorporate GAHB vectors in the remaining unconstrained dimensions. The denoiser performs a shrinkage operation on a basis adapted to the underlying image, which reveals oscillating harmonic structures along contours and inhomogeneous image regions. The paper leaves important open questions regarding the formal mathematical definition of this larger class of GAHB bases and how they result from the DNN computational architecture.
Strengths: This is a well-written paper that puts forth a compelling hypothesis.  The hypothesis is presented in a clear and concise manner, and the paper's overall readability is very good. The authors suggest that, instead of attempting to learn a low dimensional structure, DNN denoisers learn a well-regularized geometry-adaptive harmonic basis (GAHB) with small coefficients. Hence, the denoising algorithm performs a shrinkage operation on an image-adapted basis, explaining some of the impressive results that have been observed recently in the literature. 

The authors provide a counter-example to argue that if the hypothesis were not true, the DNN denoisers should perform poorly for those images for which GAHB is not the optimal basis. They construct such an example dataset using images drawn from low-dimensional structures and show that the trained DNN denoiser is not perfectly aligned with the manifold, and the bias increases with increasing noise.  Similar results were reported by creating another dataset using shuffled versions of the CelebA dataset, which would also not have GAHB as the optimal basis.
Weaknesses: The paper's results were obtained using a straightforward CNN architecture. However, prior studies have indicated that CNN architectures can effectively learn and utilize harmonic bases (https://arxiv.org/abs/1810.12136). In my opinion, the paper's only weakness lies in not acknowledging this perspective that has been previously established in the literature.
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 4 excellent