Kati8/ICLR_2023_2025 huggingface dataset:
authors: Zahra Kadkhodaie
authors: Florentin Guth
authors: Eero P Simoncelli
authors: Stéphane Mallat
reviews: 
Comments: The paper studies the inductive bias of the models which involve a denoising step, i.e., score-based diffusion algorithms. To do so, the authors take a spectral approach and analyze the eigenspace and eigenvalues of the corresponding DNN denoiser Jacobian. The authors conjecture that the DNN denoiser are implicitly biased toward so called geometry-adaptive harmonic bases (GAHBs). The authors first show that such biases emerge on a set of synthetic $C^{\alpha}$ images. Further strengthening a point,  the authors empirically show that the bias still emergences similarly even in suboptimal scenario (disk images and shuffled CelebA).
Strengths: - the paper is easy to follow and elaborate
- a good set experiments which validate the main points of paper fairly, e.g., optimal PSNR for $C^\alpha$ images and slow eigenvalues decay on CelebA
- clean mathematical framework that is used as a foundation for the empirical part 
- indication of the fact that DNNs are adapted to the high-dim structures (with certain regularity) that allows to infer them from small portions of data
Weaknesses: - further validation on a more realistic data might be beneficial to strengthen the main points of the paper
- lack of any sort of description of more "general DNN GAHBs"
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: The authors address recent concerns regarding the memorization of denoising DNNs for generative image modeling by showing that two networks trained on disjoint datasets converge to the same score/density and hence generate similar images. Those experiments are supported by theoretical derivations clearly relating the inductive bias of the denoiser to that of the density. Leveraging recent work, e.g., (Monoho et al.,2020), the authors elucidate the sparse optimal basis, adapted to the geometry of the input image, along which denoising can be understood as a shrinkage operation. Furthermore, the authors validate and connect the results by evaluating the inductive bias on image classes with known optimal bases.
Strengths: - Presents compelling empirical evidence, supported by plausible theoretical derivations, of the inductive bias of denoising DNNs.
- Explains with clear examples how they transition from memorization to generalization (even when trained on disjoint partitions of the datasets).
- Repeats the analysis on 2 datasets.
- Validates on image classes with known optimal bases.
Weaknesses: Nothing stands out, just needs to finalize the text.
Recommendation: 10: strong accept, should be highlighted at the conference
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 4 excellent