Venue: ICLR 2022 Track Blog

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Since the main motivation of the blog post is to lower the barrier to entry for readers to understand diffusion models, it is good to give an overview of what a diffusion model is and provide some background before moving on to the Intuitions section.

**Diffusion Process** it will be useful to add a visualization (e.g., a figure or an animation) of the diffusion process. The code block is not making the process easy to understand.

**Mixing Equation**: this section can be explained visually using animations. I think writing down the equation in LaTeX form can be easier to follow than having a code block.

**Algorithm**: Starting the section with the "DiffusionModel" code block can be hard to follow, especially when the parameters are not formally defined. Instead of showing the code at the beginning, it could be better to convey the high-level intuition using animations and figures. 

**Task**: the two ways discussed are very hard to follow. It will be nice to give some concrete examples and intuitions why these are the two ways to solve the problem. The code block is also hard to follow without defining the parameters. The input name should also be consistent with the ones used in the text (either image or signal). "noise_rates" and "signal_rates" are also not properly introduced. Readers who have no experience in diffusion models might wonder why we need them.

**Training**: this section is very hard to follow. There are lots of code with undefined functions and parameters. Again the parameter name in the code should also be consistent with the ones used in the text discussion. Instead of listing the code, it would be good to use animation and text to convey the high-level idea. If it is necessary to explain using code, it would be good to use a notebook format, where the output of the cell is also shown.

**Sampling**: the code is hard to follow. Can add some visualizations of each intermediate sampling step and explain the ideas based on the figures.

**Network**: the network section is too dense. Readers without background in diffusion models can find it hard to follow. Instead of listing all the details, it is important to explain the high-level intuition why these configurations are making a difference. For instance, instead of showing the implementation of "sinusoidal embeddings", explain why we need it.

**Applications**: it would be nice to organize this section using a notebook format. For instance, the reader can see the code used to generate the output. At the same time, make sure the input and function parameters are defined clearly and include a link to the notebook.

**Summary**: The current version of the blog is not very accessible to people new to diffusion models. Many sections have very dense discussions, which might only be understood by people who already had a sufficient amount of background on diffusion models. Since the main motivation of the blog post is to lower the barrier to entry for people to understand diffusion models, the blog post can be improved by incorporating more visualizations, animations, figures, intuition, and high-level discussions.




review_title: Good motivation, but the writing and structure of the blog post can be improved

final_score: rating: 5: Marginally below acceptance threshold
review_content: **Overview**: The blog post aims to illustrate a simple interpretation to Diffusion Denoising Implicit Models, with an TensorFlow implementation embedded in. While I am interested in seeing a blog post on this topic, I feel that the current version can be a bit confusing to the target audience. 

**Strengths**:

The blog post provides a simple code base in Keras that is easy to run. The topic is on diffusion generative models, which have seen good empirical performance on image synthesis and has been applied to various domains as well.

**Weaknesses**:

My understanding is that one of the goals of this blog post is to "reduce the barrier of entry to diffusion models", so we should assume that the target audience would not have much background on the topic. Unfortunately, the current version seems to be quite difficult to understand, perhaps even for readers who have familiarity over the topic and just want to understand the "novel viewpoint". This is also plagued by too much text and code as opposed to illustrative images. If the target audience had no exposure to diffusion models, I would not expect them to understand this blog post. 

My main, high-level suggestion is as follows: try to clearly define a target audience, and assume their background when writing the blog post. For someone with background on diffusion models, a lot of the things discussed in the current blog post do seem quite natural, but it might not be the case for the target audience. Detailed comments are as follows.

*Intuitions* 
- "we force these models to sample from the data distribution by misleading them.", at this point the reader would have no idea what is meant by "misleading".

*Diffusion process* 
- Noise is not discussed as "Gaussian", which is very important here to many any derivations work in this case. 
- It is unclear why we want to do this process of adding noise (and what is the diffusion timestep) in the first place (it is not the generative model). Might be worth adding an overview figure of what the model does, like in the DDPM or DDIM paper. 

*Algorithm*
- I personally feel that the Keras presentation is not necessary until the end of the blog post. Having to talk about the "model class" in the middle breaks the flow. 

*Task*
- "precisely signal-noise separation": perhaps another way that is more easily interpreted is "denoising autoencoders"? Also, might help to recall the reader that given an noisy image, the noise you predict here is not actually random, but derived from the mixing equation.
- Also, if mathjax is allowed, then consider using it when equations are discussed. 

*Training*
- Diffusion schedule is not well explained yet, how are they correlated with signal and noise rates, and why do we need them for the process?

*Sampling*
- Signal rate of exactly zero is something to be explained, but having it here could make the readers more confused. Perhaps it would be best to start with "following step" and then briefly explain the initial conditions (which is a relevant concern, but not the key insight of diffusion models). 
- The DDIM paper does not only introduce the one equation that you discussed (that is just for eta = 0); so the title and abstract seems to be slightly overclaiming. Also I think the intuition from the DDIM paper when eta = 0 is also similar to your intuition. 
- It is not explained why we are doing this (instead of some other seemingly okay updates). Personally, I feel that the most precise explanation is that *this process is the best variational approximation to a certain diffusion process*.  
- The DDIM paper and the SDE paper both discussed a "probability flow" / "neural ODE" version of the model (which are actually equivalent, except for the different solvers used, see appendix of DDIM paper). It would be nice to have it here to see why we can interpolate in the first place. 

*Stochastic sampling*
- The interpretation is exactly the same as the one in the DDIM paper, which introduced the stochastic sampling procedure first and then introduced DDIM as an limit where the procedure is deterministic (`noise_rate_multipliers` is also tied to the eta hyperparameter in DDIM). The way it is currently written may imply that the author came up with this independently. 
- "This means, that if the predicted noise is normally distributed, we remain in distribution ..." as I mentioned earlier, the "predicted noise" is not stochastic given the noisy input, and therefore cannot be normally distributed in the way it is implied (with a fixed, non-zero variance).
review_title: Interesting topic, but need to focus on making the material understandable by the target audience.

Decision: Reject