Title: Your Denoising Implicit Model is a Sub-optimal Ensemble of Denoising Predictions
Authors: Ge Kan
Jinhu Lu
Tian Wang
Lei Huang
Baochang Zhang
Aichun Zhu
Hichem Snoussi
Venue: ICLR 2023

Abstract: Denoising diffusion models construct a Markov denoising process to learn the transport from Gaussian noise distribution to the data distribution, however require thousands of denoising steps to achieve the SOTA generative performance. Denoising diffusion implicit models (DDIMs) introduce non-Markovian process to largely reduce the required steps, but its performance degenerates as the sampling steps further reducing. In this work, we show that DDIMs belong to our $\textit{ensemble denoising implicit models}$ which heavily rely on the convex ensemble of obtained denoising predictions. We propose improved DDIM (iDDIM) to demonstrate DDIMs adopt sub-optimal ensemble coefficients. The iDDIM can largely improve on DDIMs, but still deteriorates in the case of a few sampling steps. Thus we further propose $\textit{generalized denoising implicit model}$ (GDIM) that replace the ensemble prediction with a probabilistic inference conditioned on the obtained states. Then a specific instance $t$-GDIM that only depends on the latest state is parameterized by the conditional energy-based model (EBM) and variational sampler. The models are jointly trained with variational maximum likelihood. Extensive experiments show $t$-GDIM can reduces the sampling steps to only 4 and remains comparable generative quality to other generative models.


Review discussion:
The strengths:
- This work is well-motivated: to study how to improve the sampling efficiency with DDIM. This is a pressing challenge in applying diffusion models in practice.
- It is nice to see the authors point out one possible reason that DDIM could not further reduce the sampling to a few steps is the use of sub-optimal convex combination of the obtained denoising predictions.  Since these coefficients are derived by certain constraints, one natural extension is to relax those constraints thus having more flexibility. This also leads to the free-form probabilistic inference used by the proposed GDIM model.
- The image generation quality measured by FID/IS on CIFAR-10 and CelebA-64/128 are competitive with existing methods with similar or a greater number of sampling steps.
The weaknesses:
- The presentation puts almost equal emphasis on both the iDDIM and GDIM. To me, the iDDIM part is just used to illustrate the implicit dependence of all the existing states {$x_t, ..., x_T$} when peforming $f_{\theta}(x_t, t)$ to get $x_0^{t}$ and the sub-optimality of the way that DDIM leverages these states, since iDDIM just shows the existence of a better combination but does not provide an efficient way to find it.
So, this part is mainly to derive this dependence and naturally introduce the GDIM model. The space could be better used to give more detailed description of the GDIM model and the comparisons with existing models (like those discussed about recovery EBM and denoising diffusion GAN in Appendix D).
- On the other hand, the iDDIM and t-GDIM appear to be independent at the same time. iDDIM/GDIM are all about how to better leverage the information from all existing states and demonstrate solely relying on the most recent state under the equivalent form (9) is not a good idea.
But t-GDIM just considers the most recent state under the form (9), which can be seen to be independent from them.
Indeed, the authors also show that combining t-GDIM with iDDIM can provide marginal improvement in some metrics.
Also, the central idea of t-GDIM is the one introduced in Xiao et al. (2022): adding randomness instead of a deterministic mapping in $x_0$ prediction. Actually, the formulation (13) is almost identical to the equation (6) in Xiao at al. (2022) with one subtle difference: this work uses $q(x_{t-1} | x_0^t, x_T)$ where Xiao at al. (2022) uses $q(x_{t-1} | x_0^t, x_t)$. The authors have pointed out that one issue with this formulation is that the learning does not depend on $x_T$ so that the model might not fit the process as it supposes to. This might explain the inferior performance than Xiao et al. (2022).
- The arguments of the proposed method in the context of existing works appear confusing. In Appendix D, it is argued that the proposed GDIM is a more general framework than denoising diffusion GAN (DDGAN).
But it also points out the reason that the proposed method is inferior to DDGAN is due to the optimization method is similar to GAN leading to poor mode coverage in practice in Section 6.2.
However, there is no quantitative evaluation of mode coverage reported and DDGAN depends on GAN optimization as well but shows nice mode coverage properties. These raise two questions: 1) the more general framework GDIM should in theory produce better results; 2) why GAN optimization produces both poor mode coverage in GDIM and good mode coverage in DDGAN?
- It would be more convincing if the authors could provide results on more challenging high-resolution datasets like LSUN and ImageNet. It has been shown that methods that work with CIFAR-10/Celeb-64 might have a hard time with LSUN/ImageNet.
Clarity:
The paper is not hard to follow, but the presentation could be improved. Beside points mentioned above, a few minor things:
- Why is "t-GDIM" the shorthand for "radical GDIM"? What does "t" stand for?
- The "pretrained" DDPM score predictor used in Table 1 is actually the retrained version by the authors, not the pretrained DDPM checkpoint used in Watson et al. (2022). And the results for DDPM are also vastly different from those reported in Watson et al. (2022) with quadratic stride: 137.77 vs 20.225 from Watson et al. (2022) for T = 20.
- In Section 6.1, it states that "when T is larger, t-GDIM is slightly worse than FastDPM." But there are no results of t-GDIM in Table 2.
Quality and novelty:
The technical contributions appear incremental. Similar to Watson et al. (2022), this work also advocates to use all existing states but the derivation comes through the recursive expansion of the DDIM generation process.
On the other hand, the formulation of t-GDIM actually ignores all the additional dependence and follows the same route of Gao et al. (2021) and Xiao at al. (2022).
Reproducibility:
The iDDIM part should in theory be easy to reproduce. For the t-GDIM part, the authors also provide detailed information in the appendix regarding the model architectures and different hyperparameters used.
However, I am still a bit concerned due to the noticeable differences between the DDPM results reported here and those in Watson et al. (2022) and the DDIM results reported here and those in Song et al. (2021). It would be desirable if the authors would release their code and checkpoints.
NO.
We have submitted a new version of the paper. In Appendix F, we provide a comparison with DDGAN that involves most of your concern.
1) Our $t$-GDIM does has a similar spirit to DDGAN, since both of them are using some stochastic generator $G(\mathbf{u};\mathbf{x}_{t},t),\mathbf{u}\sim \mathcal{N}(0,I)$ to predict $\mathbf{x}_0$.
However, our $t$-GDIM follows a distinct theoretical route.
First of all, $t$-GDIM is based on the deterministic path in DDIM, rather than the DPM framework used in DDGAN.
Also, $t$-GDIM, as a special instance of the general GDIM, constructs a generative process to approximate the inference process (9) which is also the learning target for ensemble DIMs. The inference process (9) motivates further different generative process from that in DDIM.
So the generative process in $t$-GDIM is largely different from the DPM generative process in DDGAN. Or in other words, they leverage the denoising predictions $\mathbf{x}_0^t$ in distinct ways to generate samples.
Secondly, our GAN-based optimization is totally derived from the perspective of variational maximum likelihood, while DDGAN can not be interpreted in this framework, so the discriminator (or the energy in the paper) is different from that in DDGAN. And of course, the training method is also different in details.
In addition to the differences, GDIM is more general than DDGAN.
That is because the generative process in DDGAN is exactly following DPM. The DPM generative process is an instance of (28), and by (29), we know that DPM can also be reformulated as an ensemble model.
So GDIM, as a generalization of ensemble model, is naturally more general than DDGAN. Please see Appendix F.
2) The phenomenon of mode coverage comes from the variational maximum likelihood in theory. Note the discriminator (or energy) in $t$-GDIM $E(\mathbf{x}_0^t,\mathbf{x}_t,t)$ is defined over the space of sample $\mathbf{x}_0$, and $\mathbf{x}_T$ are almost pure noises, so the $t$-GDIM is trained like a pure unconditional GAN at step $T$. It suffers from the poor mode coverage, while entropy of the sampler can help in theory.
Though DDGAN ignores the entropy too, its discriminator is trained to distinguish the $t-1$ step noisy states.
In other words, DDGAN adds noises to the sample, which can be regarded as a data augmentation trick to compel the sampler to perform better mode coverage. Please see Appendix F.2.
3) Our work introduces two closely related framework based on the DDIM, and then propose two instances of them respectively. The first one can be applied to pretrained score model and the second one needs to be trained with GAN-based optimization. The ensemble DIM and the GDIM are closely related, because they all regard the new inference process (9) as the learning target, so they are all following the same procedure: first get a prediction $\bar{\mathbf{x}}_0^t$ and then combine it with $\mathbf{x}_T$ to obtain the next state. The first one, ensemble DIM, provides a flexible way to tune the trust on obtained denoising predictions $\mathbf{x}_0^{t:T}$ to get better $\bar{\mathbf{x}}_0^t$, and its instance iDDIM improves significantly on DDIM and even better than som recent accelerating methods.
The ensemble DIM framework is designed for the pretrained DDPM score model, and can be regarded as a sampler for it. The second one, GDIM, wants to get the prediction $\bar{\mathbf{x}}_0^t$ in a more general probabilistic way. It replaces the convex ensemble in ensemble DIM with a probabilistic inference.
So we think GDIM as a more general framework than ensemble DIM, but unlike ensemble DIM, GDIM ignores the dependence on all the previously obtained denoising predictions $\bar{\mathbf{x}}_0^{t+1:T}$ and only trust the most recent one $\bar{\mathbf{x}}_0^{t}$ at step $t$.
In other words, GDIM is no longer an ensemble model in theory and the reason is natural: the ensemble operation is not probabilistic.
As a result, iDDIM can only be regarded as an augmentation trick on GDIMs.
In fact, we can not easily interpret the $t$-GDIM+iDDIM from the probabilistic theory of variational inference given (9).
Moreover, the GDIM needs to be trained from scratch and its instance $t$-GDIM achieves competitive performance among SOTA generative models.
4) Our difference from Watson et al. (2022) also comes from the learning target (9) like that from DDGAN. Note in Watson et al. (2022), obtained states $\mathbf{x}_{t:T}$ are combined to get the next state, however, we show $\mathbf{x}_k$ are combination of $\mathbf{x}_0^{k:T}$ and noises (see Eqn. (29)). Ensemble DIM presents a framework that only depends on $\mathbf{x}_0^{t:T}$ and $\mathbf{x}_T$, so it is more general than Watson et al. (2022), and explores deeper on how DDPM/DDIM construct the denoising process.
Though we just tune the $m$ manually while Watson et al. (2022) learns the best coefficients, we find our iDDIM achieves competitive or even better performance in Tab. 2, under similar experimental setting.
5) Our $t$-GDIM is inferior than DDGAN on CIFAR10, but performs marginally better on CelebAHQ-256, please see Tab. 5 in the newest version of the paper for result.
6) The $t$ means that the probabilistic inference in $t$-GDIM only depends on $\mathbf{x}_t$. The score model used in Sec 6.1 is a retrained version, we have corrected the relevant descriptions. The DDPM experiments in Tab. 1 with quadratic stride follows the $\hat{\sigma}$ setting in DDIM (see the Tab.1 in the DDIM paper [1]), and our result 137.77 for $T=20$ is close to the corresponding result 133.37 in DDIM.
While the results 20.225 in [2] corresponds to the $\eta=1.0$ setting in DDIM [1] which gets 18.36 FID. We have corrected the "when T is larger, t-GDIM is slightly worse than FastDPM." to "when T is larger, iDDIM is slightly worse than FastDPM."
7) We have submitted our supplementary codes. Hope it will help you to understand our methods.
8) Thanks for your advice for the structure of this paper. We are making efforts to improve this work in the next version.
[1] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. International Conference on Learning Representations, 2021.
[2] Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. Learning fast samplers for diffusion models by differentiating through sample quality. In International Conference on Learning Representations, 2022.
STRENGTHS:
* I think this is a strong paper from the experimental point of view. I found Table 1 to be quite telling. It really illustrates the benefit of choosing $m \neq 0$ for a small budget of sampling steps in diffusion models.
* I also appreciated the pedagogical introduction of DDIM as well as the introduction of the ensemble method. The method seems quite natural as the authors introduced it.
* I also want to emphasize the extensive comparisons of the authors with other state of the art acceleration methods in diffusion models or improved samplers but also with GANs.
WEAKNESSES:
* While I am quite convinced by the part on iDDIM. I think the authors could do a much better at comparing themselves with [1]. To me the method introduced in the probabilistic version of generalized DDIM is quite similar to the one introduced in [1] especially the method relying on a GAN for obtained a sampler of the reconstruction. I think a deeper comparison with [1] should be conducted.
* I understand that theory of ensemble method and theory of DDIM is hard but the method seems to be more like a trick than a principled method. I would argue that many other choices are possible (for example one could consider extrapolation and not necessarily convex combination, putting even negative weights on certain samples). Is there any theoretical result backing up the specific choice of ensemble chosen by the authors.
* The issue regarding mode coverage is interesting but only discussed in the appendix. I think the authors should highlight more these limitations in the main text as these are important to understand the proposed method.
* In some table (Table 1 and Table 2) it would be great to have confidence intervals to assess if the obtained results are significant or not.
* Figure 1 (left) was not particularly helpful in its present form. I had a hard time understanding it (the equations were clearer to me). I think that a good figure could indeed explain the method quite well but as of now I find it more confusing than helpful.
OTHER QUESTIONS/COMMENTS:
* Recent results [2] suggest that knowledge distillation seems to be key to achieve very low NFE with good quality. Could the approach of the authors be combined with the approach of [2] for instance? I understand that [2] appeared after the paper was submitted and so I don't claim that this is a weakness of the paper. I am, however, curious about the combination between ensemble DDIM and distillation.
[1] Xiao, Kreis, Vahdat - Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
[2] Meng, Gao, Kingma, Ermon, Ho, Salimans - On Distillation of Guided Diffusion Models
The paper is quite clear.
The proposed methodology is a simple, yet efficient, extension of DDIM.
The paper is well-written and quite polished, especially regarding the experiments
The authors provide experimental details to reproduce the experiments
NO.
No ethical concern.
1) Our $t$-GDIM does has a similar spirit to DDGAN, since both of them are using some stochastic generator $G(\mathbf{u};\mathbf{x}_{t},t),\mathbf{u}\sim \mathcal{N}(0,I)$ to predict $\mathbf{x}_0$.
However, our $t$-GDIM follows a distinct theoretical route.
First of all, $t$-GDIM is based on the deterministic path in DDIM, rather than the DPM framework used in DDGAN.
Also, $t$-GDIM, as a special instance of the general GDIM, constructs a generative process to approximate the inference process (9) which is also the learning target for ensemble DIMs. The inference process (9) motivates further different generative process from that in DDIM.
So the generative process in $t$-GDIM is largely different from the DPM generative process in DDGAN. Or in other words, they leverage the denoising predictions $\mathbf{x}_0^t$ in distinct ways to generate samples.
Secondly, our GAN-based optimization is totally derived from the perspective of variational maximum likelihood, while DDGAN can not be interpreted in this framework, so the discriminator (or the energy in the paper) is different from that in DDGAN. And of course, the training method is also different. We provide a detailed comparison in Appendix F in the newest version of the paper.
2) In this work, the ensemble DIM comes from another understanding of the DDIM formulation, and we are exploring the nature of how DDIM constructing generative process. The ensemble DIM is a generalization of DDIM that allows us to tune the ensemble coefficients flexibly. Generally speaking, the central theory comes from the form (9). Since the form (9) actually depicts the deterministic path from $\mathbf{x}_0$ to $\mathbf{x}_T$, and the path is a linear combination of $\mathbf{x}_0$ and $\mathbf{x}_T$ with positive coefficients.
So we think the ensemble DIM should only use convex combination in theory.
However, the core problem in ensemble DIM is to leverage $\mathbf{x}_0^{t:T}$ to obtain better $\bar{\mathbf{x}}_0^t$.
So it is of course ok to perform extrapolation in practice as long as it can provide better $\bar{\mathbf{x}}_0^t$.
3) The phenomenon of mode coverage comes from the variational maximum likelihood in theory, we provide a detailed discussion on it in Appendix F.2 in the newly submitted paper. We will try to highlight more these limitations in the main text in next version.
4) We design the structure and the components in Tab 1 and Tab 2 following the DDIM paper (see Tab 1 in [1]).
5) Sorry for the confusing figure, we are trying to make it more helpful in the next version.
6) Please see Tab 5 in the new version of paper for the results on CelebAHQ-256. Our $t$-GDIM performs better than DDGAN and some other SOTA models on higher resolution.
7) We have submitted our supplementary codes. Hope it will help you to understand our methods.
[1] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. International Conference on Learning Representations, 2021.
Strength: This paper provides a theoretical perspective that DDIM is a specific case of the ensemble denoising implicit model with convex coefficients.
Weaknesses: The experiment section can be improved in the paper. Please refer to the details in the next section.
1. In Proposition 1, the denoising diffusion implicit models (DDIM) are equivalent to improved DDIM (iDDIM). It is not clear why iDDIM performs better than DDIM. Could you provide more discussions on this?
2. In the experiment, the authors set $m_t=m$ for all t in iDDIM. According to Eqn. (7), the coefficients $m_t$ are different. It would be better to discuss the disadvantages of this setting.
3. The experiments are only conducted on a simple dataset (CIFAR100) and CelebA with a small resolution. To make the experiment convincing, it would be better to conduct more experiments on other datasets or high resolution.
4. It is expensive to tune the best $m^*$ in the experiment.  Is it possible to learn the best $m^*$ with a neural network?
NO.
1) DDIM is not equivalent to iDDIM. In this work, we first introduce the general ensemble DIM (EDIM) (4) which allows us to tune the ensemble coefficients $\omega_t^k$ before obtained denoising predictions $\mathbf{x}_0^{k}$ for better prediction of $\mathbf{x}_0$.
Then we prove that DDIM (6) is an instance of the general EDIM in Proposition 1.
Further, we introduce the radical EDIM (10) which is also an instance of the general EDIM. At last, we prove that the interpolation between DDIM (6) and radical EDIM (10), i.e. the iDDIM (11), is also an instance of EDIM.
So iDDIM is not equivalent to DDIM, but they are all belong to our proposed new general class (4). The reason for why iDDIM performs better than DDIM is intuitively explained in Fig. 1, and is experimentally explained in Fig. 4. The Fig. 1 shows that, at step $t$, the most recently obtained denoising prediction $\mathbf{x}_0^t$ is intuitively more precise than previous obtained denoising predictions $\mathbf{x}_0^{t+1:T}$ (this is experimentally verified in Fig. 4).
So we can get better prediction of $\mathbf{x}_0$ by appropriately turning up the coefficient before $\mathbf{x}_0^t$.
That is how iDDIM improve on DDIM, and the experiments show that iDDIM indeed outperforms DDIM by a large margin when the number of sampling step $T$ is small.
2) The coefficients $m_t$ introduced in iDDIM (11) is used to tune the interpolation between DDIM (6) and radical EDIM (10).
In other words, it controls the ratio between the most recent denoising prediction $\mathbf{x}_0^t$ and the previous obtained denoising predictions $\mathbf{x}_0^{t+1:T}$ in the sense of ensemble. In the general case, $m_t$ are different at different steps $t$.
But we set $m_t=m$ for all $t$ in the paper.
In theory, this setting is less flexible than the general case.
However, we find it is good enough in experiments, and we do not need to handle many $m_t$ for efficiency.
3) Experimentally, it is not very expensive to find a good $m$ since the number of steps $T$ is much smaller than that in standard diffusion models.
On the other hand, it is possible to learn the best $m^*$ by neural networks.
In fact, we may leverage the idea of differentiating through sample quality [1] to accomplish this.
However, in Tab. 2, we find our iDDIM achieves competitive or even better performance than [1], under similar experimental setting, though we just tune the $m$ manually.
4) Please see Tab. 5 in the newest version of paper for the results on CelebAHQ-256. Our $t$-GDIM performs competitively or better among some other SOTA models on higher resolution.
5) We have submitted our supplementary codes. Hope it will help you to understand our methods.
[1] Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. Learning fast samplers for diffusion models by differentiating through sample quality. In International Conference on Learning Representations, 2022.
## Strengths
The paper provides a nice conceptual framework for understanding DDIMs which clearly motivates alternative weighting schemes, iDDIM.
The proposed iDDIM weighting schemes (favoring recent samples) improves generation with few time steps.
## Weaknesses
Superficially at least, the proposed GDIM method seems quite similar to DDGAN, but performs worse.
While the appendix has discussion of the relationship between the two methods (which likely belongs in the main text) it's unclear what the practical distinction between the two methods is. The main text states only that DDGAN has a "similar spirit to t-GDIM, but ours follows a distinct theoretical route." For some reason DDGAN was not included in the CelebA comparisons -- it would have outperformed the proposed method (which appears as SoTA in the table) by a large margin.
Clarity: The paper can be hard to follow and then English could use some work. There's a jump in going from iDDIM to GDIM, which does not seem well-motivated nor well-explained.
Quality: While the reweighting scheme improves DDIM performance (Table 1), the proposed method seems strictly worse than the closely related DDGAN method.
Novelty: While the iDDIM contribution is interesting and I believe novel, the t-GDIM method seems to overlap heavily with DDGAN.
Reproducibility: The proposed method seems quite difficult to replicate, there's no supplementary code, and there's no indication code will be released upon acceptance.
NO.
1) Our $t$-GDIM does has a similar spirit to DDGAN, since both of them are using some stochastic generator $G(\mathbf{u};\mathbf{x}_{t},t),\mathbf{u}\sim \mathcal{N}(0,I)$ to predict $\mathbf{x}_0$.
However, our $t$-GDIM follows a distinct theoretical route.
First of all, $t$-GDIM is based on the deterministic path in DDIM, rather than the DPM framework used in DDGAN.
Also, $t$-GDIM, as a special instance of the general GDIM, constructs a generative process to approximate the inference process (9) which is also the learning target for ensemble DIMs. The inference process (9) motivates further different generative process from that in DDIM.
So the generative process in $t$-GDIM is largely different from the DPM generative process in DDGAN. Or in other words, they leverage the denoising predictions $\mathbf{x}_0^t$ in distinct ways to generate samples.
Secondly, our GAN-based optimization is totally derived from the perspective of variational maximum likelihood, while DDGAN can not be interpreted in this framework, so the discriminator (or the energy in the paper) is different from that in DDGAN. And of course, the training method is also different. We provide a detailed comparison in Appendix F in the newest version of the paper.
2) Our $t$-GDIM is inferior than DDGAN on CIFAR10, but performs better on CelebAHQ-256, please see the newly submitted paper for some results. We do not compare our model with DDGAN on CelebA-64 and CelebA-128 because they do not report their results on these datasets.
3) Our work introduces two closely related framework based on the DDIM. This is because, the ensemble DIM and the GDIM all regard the new inference process (9) as the learning target, so they are all following the same procedure: first get a prediction $\bar{\mathbf{x}}_0^t$ and then combine it with $\mathbf{x}_T$ to obtain the next state. The first one, ensemble DIM, provides a flexible way to tune the trust on obtained denoising predictions $\mathbf{x}_0^{t:T}$ to get better $\bar{\mathbf{x}}_0^t$, and its instance iDDIM improves significantly on DDIM and even better than som recent accelerating methods. The ensemble DIM framework is designed for the pretrained DDPM score model, and can be regarded as a sampler for it. The second one, GDIM, wants to get the prediction $\bar{\mathbf{x}}_0^t$ in a more general probabilistic way. It replaces the convex ensemble in ensemble DIM with a probabilistic inference.
So we think GDIM as a more general framework than ensemble DIM, but unlike ensemble DIM, GDIM ignores the dependence on all the previously obtained denoising predictions $\bar{\mathbf{x}}_0^{t+1:T}$ and only trust the most recent one $\bar{\mathbf{x}}_0^{t}$ at step $t$.
In other words, GDIM is no longer an ensemble model in theory and the reason is natural: the ensemble operation is not probabilistic.
As a result, iDDIM can only be regarded as an augmentation trick on GDIMs.
In fact, we can not easily interpret the $t$-GDIM+iDDIM from the probabilistic theory of variational inference given (9).
Moreover, the GDIM needs to be trained from scratch and its instance $t$-GDIM achieves competitive performance among SOTA generative models.
4) We have submitted our supplementary codes. Hope it will help you to understand our methods.

Meta-review:
role: PC
data: metareview:_summary,_strengths_and_weaknesses: The paper proposes to view DDIM as a special case of ensembling previous predictions that have suboptimal ensemble coefficients. It shows that DDIM can be improved with a better designed set of ensemble coefficients. However, improving the ensemble coefficients is still not good enough given a few number of sampling steps, which motivates the authors to propose generalized denoising implicit model (GDIM). GDIM is closely related to DDGAN, but overall produces inferior results.

Overall, the reviewers recognize the potential values of viewing DDIM as a suboptimal ensemble of denoising predictions, but there are several notable weaknesses. Combining the reviews and his/her own reading, the AC summarizes the main weaknesses as follows:

1. The proposed two models, iDDIM ad GDIM, are not tightly connected to each other, and GDIM is somewhat drifting away from the main motivation of improving the ensemble of denoising predictions.

2. Requiring the tuning of "m" for each dataset and each T makes iDDIM not user-friendly.

3. GDIM is closely related to DDGAN but performs worse.

4. Lack of theoretical understandings of why an improved ensembling method would help improve DDIM.
justification_for_why_not_higher_score: The paper needs a major revision to address the listed weaknesses.
justification_for_why_not_lower_score: N/A

Decision: Reject