Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces the wavelet diffusion model, which is theoretically guaranteed to achieve $\epsilon$ error, independent of the step size.
strengths_and_weaknesses: $\textbf{Strengths}$
- This work provides theoretically and empirically sufficient evidence that the wavelet diffusion model has a fixed number of function evaluations to synthesize an image, independently of the resolution.
- In particular, the theory is excellent, and the arguments are solid.

$\textbf{Weaknesses}$
- In contrast to the theory, the experiments are disappointing. The authors only demonstrate their claim on the CelebA-HQ dataset, which significantly differs from the ImageNet or CIFAR-10 dataset in its diversity. If the suggested wavelet diffusion model is truly effective, then at least the authors should have proved its efficacy on various datasets.
- The number of sampling iterations to reach an error $\epsilon$ is independent of the dimension only for the case of the Gaussian process. I'm a fan of this paper, but the lack of the analogous proof of Theorem 2 to the wavelet diffusion model is a bit unsatisfactory, though I understand how difficult it is to prove.
- Though the paper has a clear and solid lesson, it is still largely missing how to use this lesson in practice. For instance, it would be much better to compare the vanilla SGM and the WSGM when the number of discretization increases to $O(1000)$. I suspect that SGM outperforms WSGM as $L>1000$, is it right? If then, the authors should clearly point out that as their limitations.
questions: -
limitations: -
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes Wavelet score-based generative model (WSGM), a multi-scale score-based generative model with an accelerated sampling by reducing the number of required iterations of the score estimator leveraged by renormalized wavelet coefficients. An important property of WSGM is a theoretical analysis on the required number of sampling iterations to reach the desired error which is not dependent on the dimension of the signal per scale.
strengths_and_weaknesses: ### Strengths
The paper presents a rigorous theoretical proof and mathematical analysis on the connection between the normalized wavelet coefficients and the sampling iterations of SGM. The proposed method, the renormalized wavelet decompositions, provides a principled approach to reduce the Lipschitz irregularity [De Bortoli+2021] from the score estimator.

The experimental results on three classes (Gaussian process, physical process, and natural images) support the theoretical foundation of WSGM, where it surpasses the baseline multi-scale SGM under the same number of discretized reverse diffusion steps. Although the considered baseline model is not based on recent state-of-the-art SGM (uniform discretization for sampling, for example), the controlled experiments in a principled way are convincing enough for me to justify the rigorous theoretical contribution of WSGM.

### Weaknesses
Although the wavelet decomposition-based approach is well motivated with mathematical connections, I would like to point out that such approach for the multi-scale generative model has also  been studied, such as Wavelet Flow [Yu+2020] on normalizing flows, or SWAGAN [Gal+2021], for example. Although the main scope of this paper is SGM, I suggest allocating paragraphs for a description of related work using wavelet approach for the multi-scale generative model, to present a complete picture on the considered approach.

The current scope of empirical experiments is sound, but I consider it a missed opportunity to show the result on WSGM on the high-resolution images. Considering the recent progress on SGM on high-fidelity synthesis of images, WSGM is expected to have a potential to achieve a high-resolution image modeling (1024x1024 for example) which is currently not easily approachable. In fact, [Yu+2020] showcased their method as the first to unlock model training and sampling on 1024x1024 images. I believe that additional experiments on the high-resolution images will strengthen the impression of WSGM also from the practical point of view. 
questions: Understanding that the high-resolution image experiments may be challenging during the discussion period, do the authors have an opinion on expected outcome? Are there practical blockers of WSGM for the scaled-up evaluation? I believe that a high-impact work related to the diffusion based models often comes with the promising empirical studies using real-world high-resolution datasets.

Accelerating SGM inference is one of the most actively studied area, which naturally deviates from the controlled experimental setup of WSGM (uniform discretization). Can the authors consider more realistic experimental setups using several popular acceleration methods? What will be the implication from such assessment?
limitations: The authors have addressed the limitation of WSGM, that is, using uniform discretization schedule for SGM sampling which does not represent modern SGMs.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors attempted to alleviate the high computation cost in Score-based generative models (SGMs) by reducing the number of time steps. In particular, the authors first found that the number of times steps in SGMs to reach a fixed error depends on the condition number of the Gaussian covariance. To break this dependence, they proposed a novel SGMs with wavelet whitening. Solid theoretical and technical contributions have been witnessed in this paper.
strengths_and_weaknesses: Strengths:

1.The authors provide a theoretical analysis of how the regularity of the stein score affects the discretization of generation. They found that, to reach the fixed error, the number of time steps increases with the condition number of the covariance. 

2.It is an interesting and novel idea to utilize the wavelet orthogonal transform to decompose and subsample the input image, which can reduce the image size.

3.The authors prove that the proposed model can fix the same discretization schedule for all image scales and accelerate the generation in SGMs, where the time complexity grows linearly with the image size.

Weakness:

1.The authors should provide more descriptions of the wavelet transforms in this paper. It is hard for me to understand the major idea in this paper before learning some necessary knowledge about wavelet whitening, wavelet coefficient, and so on.

2.It is better for authors to display the performance of accelerating SGMs by involving some other baselines with a different perspective, such as “optimizing the discretization schedule or by modifying the original SGM formulation” [16, 15, 23, 46, 36, 31, 37, 20, 10, 25, 35, 45] 

questions: Is the proposed model competitive compared with other baselines to accelerate SGMs, such as the models in referred paper "[16, 15, 23, 46, 36, 31, 37, 20, 10, 25, 35, 45] "? 
limitations: None
review_title: null

Meta-review: metareview: Summary: This paper introduce Wavelet Score-Based Generative Modeling, a multi-scale diffusion models that allows for considerably faster synthesis by working in the wavelet basis. The topic is timely, given the community's growing interest in diffusion models, and the high computational cost of sampling from such models. The reviewers think that this is a technically solid paper that introduces important insight, for example that the number of required time steps increases with the condition number. The reviewers found the empirical comparison to a baseline diffusion model satisfactory The reviewers had some relatively minor concerns, some of were addressed in a healthy discussion. For example. the submitted paper lacked an introductory explanation of the wavelet transform, but this was addressed by the authors by including an additional appendix. The reviewers agree that the theoretical and empirical contributions are sufficient for publication. The reviewers do note that experiments on larger-scale and/or more diverse datasets (e.g 1024x1024) would be illuminating. There was no discussion among reviewers.

Recommendation: I recommend to accept this paper. 
confidence: Certain
Decision: Accept