From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Accept: poster
metareview:_summary,_strengths_and_weaknesses: The paper first shows that the inverse heat dissipation model can be considered a non-isotropic diffusion process in the spectral space. It then presents a generative model that is inspired by this realization. The generative model borrows the strength from DDPM and the idea from the inverse heat dissipation work. The presented results verify that the presented generative model works.

The paper received a mixed rating. Two reviewers consider the paper above the bar, citing the idea as interesting and the improvement over the inverse heat dissipation model major. The two other reviewers consider the paper slightly below the bar, citing issues including insufficient experiment results and unclear technical contribution. This puts the paper in the borderline category.

The AC conducted an AC-reviewer meeting. The strength and weaknesses of the paper were discussed intensively in the meeting. Based on the arguments made in the meeting, the AC is in favor of accepting the paper. First, the presented connected to the inverse heat dissipation model is interesting. Second, the presented algorithm seems to be working. While the experiment and presentation quality could be improved, the paper is interesting enough for the ICLR community.
justification_for_why_not_higher_score: The experiment results are not strong enough to prove this method will change the way people implement diffusion models.
justification_for_why_not_lower_score: The experiment results support the proposed idea, which is interesting and contemporary.
summary_of_AC-reviewer_meeting: 3 out of 4 reviewers attended the meeting. The one, who could not make it, needs to attend a last-minute meeting. 

In the AC-reviewer meeting, we asked each reviewer to comment on the number one weakness and then the number one strength. From the comments, we tried to gauge better where this paper lies. In the weakness comment round, the major weakness includes that the paper could present more results, and the paper looks like a last-minute rush. In the strength comment round, the major strength includes the presented idea being interesting enough and the presented results validating the proposed idea.

The AC took a stance and argued the paper was above the bar and asked for arguments to reject the paper. No convincing arguments against the acceptance were given.
note_from_PC: if the above contains the word "oral" or "spotlight" please see: "oral" presentation means -> notable-top-5% and "spotlight" means -> notable-top-25%. As stated in our emails, we are disassociating presentation type from AC recommendations

From: ICLR.cc/2023/Conference/Paper2519/Authors
title: location of ablations
comment: The results had been added to Appendix B. We also included a description of the frequency experiment now.

From: ICLR.cc/2023/Conference/Paper2519/Reviewer_oaCX
title: Where to find new results (ablation outcomes)?
comment: Can you please point us to the places in the main text or supplement where you report on the outcome of the discussed ablations?
Thanks!

From: ICLR.cc/2023/Conference/Paper2519/Authors
title: author response
comment: We thank the reviewer for their time and address the raised questions below:

1. We have added the technical implications of our work at the end of section 2. Essentially: our derivation ensures that we have the same correctness guarantees that we have in standard diffusion, meaning that that the model p(z_s |  z_t) is able to learn q(z_s | z_t) → q(z_s | z_t, x=E[x | z_t]) as s approaches t, because now we have been able to derive the exact (non-isotropic) noise distribution of q(z_s | z_t, x). Hopefully this clears up the connection with Rissanen et al, (2022). 

2. On the comparison with Ho et al., (2020): In their paper a different architecture is used that has one more level, albeit with less residual blocks. Furthermore, perhaps less obviously, they use a clipping heuristic that improves FID. In this paper, we aimed for a fair comparison and simply trained a diffusion model with the same settings without any further heuristics. Regarding the code, even though we do not plan to release the code, many great open-source implementations exist. In addition, Appendix A provides the necessary computations that are specific to diffusion with blurring, and includes precise details such as how we use small numerical values for numerical stability of division.

From: ICLR.cc/2023/Conference/Paper2519/Authors
title: author response
comment: We thank the reviewer and address the points below:
1. On polishing the related work, we have added several more references. Please let us know if there are others that should be added.

2. On the experimental results. We have turned “interpolation” off for pdf-viewers and fixed the cifar10 png file, which means that for both figures they should now be correctly rendered. Latex uses lossless pngs that exactly match the required pixels. Unfortunately, pdf-viewers can decide themselves to still render certain images with interpolation (e.g. macOS preview does this), and there is not another way to enforce this behavior. Let us know if the issue persists and we will find another way to send the images anonymously. We have also computed and reported Inception scores for the CIFAR10 experiment in the Appendix.

From: ICLR.cc/2023/Conference/Paper2519/Authors
title: author response
comment: We thank the reviewer for their time and address the raised weaknesses below:

4. On positioning the paper with respect to existing works. The paper was intended to bridge the gap between the knowledge that already exists for denoising diffusion, and the new emerging field of different (correlated) diffusion processes, with emphasis on blurring. The positioning was chosen to be in direct relation to these methods, to clear up possible misconceptions about using blurring as a diffusion model.

5. We have expanded on the “lack of guarantees of convergence to the data distribution” at the end of section 2. Essentially, the point is that if p(z_s | z_t) is of the form q(z_s | z_t, x=x_prediction) (which we derive is non-isotropic) then the model can approximate q(z_s | z_t) arbitrarily accurately as s tends to t. With a different specification, such as the one in Rissanen et al., this is generally not possible. We have also added a sentence on the regularization in the first paragraph of section 4.

6. We have gone over the text to improve readability and to fix (in-line) citations. Please let us know if there are still issues that persist in this respect.

From: ICLR.cc/2023/Conference/Paper2519/Authors
title: author response
comment: We thank the reviewer for their time, in the following we address the weaknesses:

1. On x-parametrization, we have run an ablation using x-parametrization on CIFAR10 for the best performing model, and achieve 23.9 FID versus 3.17 with eps-parametrization. 

2. On pixel space vs frequency space, we have run an experiment on frequency space. We can see that the model is not able to learn the task correctly and samples are grey (for completeness, 260.5  FID). Also, we have rephrased the section slightly which now reads: “neural networks have been optimized to work well in standard pixel space”

3. We added the consequences of these contributions to the end of section 2. In short: it allows us to analytically derive the required noise instead of heuristically choosing it. Furthermore, this ensures the same correctness we have in standard diffusion. Meaning that the model p(z_s |  z_t) is able to learn q(z_s | z_t) → q(z_s | z_t, x=E[x | z_t]) as s approaches t.

From: ICLR.cc/2023/Conference/Paper2519/Reviewer_oaCX
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The presented work establishes a theoretical link between standard diffusion models and Inverse heat dissipation models (IHDMs). They show that IHDM formulation can be equally represented as a standard diffusion model, albeit in the frequency domain. They use this theory to apparently get the best from both approaches: (a) They are able to work with blurring which standard diffusion models are not, and (b) use epsilon parametrization which has been empirically shown to outperform the basic representation for standard diffusion models. IHDM could not use it since it was not a diffusion model. 
The utility of the presented approach is demonstrated on two datasets, improving the performance of heat dissipation models significantly. 
strength_and_weaknesses: Strength:
1. The authors derive an equivalence between IHDMs and standard diffusion models.
2. This is later used to significantly improve the performance of heat dissipation based models. 
3. Other theoretical contributions: a) heat dissipation can be a markov process by explicitly giving one formulation, and (b) inverse heat dissipation, unlike inverse diffusion,  is not an isotropic process (in the frequency domain).

Weaknesses:
1. In pixel space, it has been shown that epsilon formulation is better. It would be a good ablation to show this holds in the frequency domain as well.
2. Page 5 last line: “ …it is convenient to express our diffusion and denoising processes in frequency space, neural networks tend to operate well on a standard pixel space”. Either there should be references supporting this or even better, an ablation study showing that neural networks perform better on pixels. 
3. The significance of the two theoretical contributions namely a) inverse heat dissipation not being isotropic and b) possibility of heat dissipation being a markov process is not discussed. 
clarity,_quality,_novelty_and_reproducibility: This paper is clearly written and has both theoretical and empirical contributions. 

summary_of_the_review: A good quality paper which theoretically links heat dissipation models with diffusion models and use that to improve the heat dissipation model. Some more work can be done on ablation studies. 

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper

From: ICLR.cc/2023/Conference/Paper2519/Reviewer_6MNZ
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: A paper bringing together aspects from diffusion (noising) and heat dissipation (blurring)  in a best-of-both-worlds fashion. Proof-of-concept results deliver impressive image quality.
strength_and_weaknesses: *Strengths:*

1. This is an interesting paper that bridges the gap between inverse heat dissipation (recently proposed by Rissanen et al.) and denoising models. The novelty lies in formulating the (de)blurring model as a Gaussian diffusion process with non-isotropic noise. This formulation is novel and complements the work by Rissanen.

2. The experiments are brief, but demonstrate the strengths of the proposed approach. The visual quality of the generated samples (as well as FID scores) is impressive---especially if compared to that given by vanilla IHDM.

3. The paper is well-structured and easy to follow. 

*Weaknesses:*

4. The paper builds directly on Rissanen et al. (2022) for the deblurring part and Song's and Ho's work on denoising. The paper is written almost more as a technical note or technical comment on these existing works than an actual full paper. This, however, makes it easy to position the paper and follow the idea. 

5. The paper focuses largely on technical details, and it would have benefited from expanding on discussion points (e.g., the regularisation effect briefly covered in Sec. 6 and 7). Points on the lack of 'guarantees of convergence to the data distribution' (Sec. 1) appear vague and could be either expanded, removed, or rephrased. Additional discussion on the role of the neural network architecture and its role would have been interesting.

6. Even if the presentation itself is clear, it appears slightly sloppy in parts, or written in haste. I would the authors to carefully go through the paper and fix in-text citations to read well.
clarity,_quality,_novelty_and_reproducibility: The idea is basically to restructure Rissanen's work and combine it with denoising diffusion models. Even so, the idea is novel and interesting.

For comments on clarity and things to improve, see 'weaknesses' above. 

The results appear reproducible, even if no actual code (beyond the pseudo-code in the appendix) was shared as part of the submission. Do the authors plan to release code to reproduce their experiments?
summary_of_the_review: This is a paper that is by no means perfect, but should be of interest to the audience of ICLR. The idea in itself is simple and extends the recent work by Rissanen, but it is a novel and impactful one.

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper2519/Reviewer_PRJZ
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper proposes a Blurring Diffusion Model. It is a type of diffusion model based on heat dissipation, or blurring. The model obtains better performance compared with some SOTA models (e.g., Denoising Diffusion, IHDM).
strength_and_weaknesses: Strength:
(1) It makes sense for combining heat dissipation and blurring into the diffusion model.
(2) The model can generate images with lower FID.

Weaknesses:
(1) The related work part needs to be polished. The previous works are not sufficient for heat dissipation, blurring, Gaussian noise, etc.
(2) The experimental results are not stable.
  a. I can not view the generated images with high resolution.
  b. The FID is the only metric used in this paper.
  c. There is no user study. 
clarity,_quality,_novelty_and_reproducibility: Clarity: 3/10, 
Quality: 4/10, 
Novelty: 5/10,
Reproducibility 6/10.
summary_of_the_review: The idea of the blurring diffusion model is good. And the FID values show that the proposed model has better performance than previous models. While the paper needs to be polished, especially for the experiment part.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper2519/Reviewer_HZtN
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper analyzes the connection between two diffusion models of DDPM and IHDM. Showing that the heat dissipation process is a Gaussian diffusion one in the frequency domain. Based on this insight, the authors propose the “blurring diffusion model”, which explicitly defines latent variables of the intermediate diffusion steps in the frequency space. Experiments show comparable results with the DDPM and improved ones compared with IHDM. 
strength_and_weaknesses: The major contribution, to the best of my understanding, lies in the analysis of IHDM drawing its connection to denoising diffusion in frequency space, as well as the proposed latent variable model. Another interesting insight is that other deterministic transformations may be also applicable for destroying the signal. 
However, I’m not completely sure about the significance of the contribution, especially in comparison with Rissanen et al. 2022. Perhaps I missed something important but it would be great if the authors could highlight the technical improvement / difference. 
I’m not sure if the experimental result is convincing enough either. Checking the Ho et al. 2020, I realized that their simple loss already yielded an FID of 3.17 on CIFAR10, which is exactly the same as the reported performance of the blurring diffusion model. 

clarity,_quality,_novelty_and_reproducibility: The connection with Ho et al 2020 is clear but not with Rissanen 2022.  
The proposed insight, i.e. the frequency domain aspect on Rissanen is novel to me. 
The authors gave no information whether the codes will be made available upon publication. 

summary_of_the_review: In general this is a piece of interesting insight but the presentation could have been better. The distinguishment to most related work is not clear to me and I wish that the authors could address my questions in the section above. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold