Diffusion Models as Cartoonists: The Curious Case of High Density Regions

Authors: Rafal Karczewski, Markus Heinonen, Vikas Garg
Primary area: generative models
Keywords: diffusion models, likelihood estimation

Abstract:
We investigate what kind of images lie in the high-density regions of diffusion models. We introduce a theoretical mode-tracking process capable of pinpointing the exact mode of the denoising distribution, and we propose a practical high-density sampler that consistently generates images of higher likelihood than usual samplers. Our empirical findings reveal the existence of significantly higher likelihood samples that typical samplers do not produce, often manifesting as cartoon-like drawings or blurry images depending on the noise level. Curiously, these patterns emerge in datasets devoid of such examples. We also present a novel approach to track sample likelihoods in diffusion SDEs, which remarkably incurs no additional computational cost. Code is available at https://github.com/Aalto-QuML/high-density-diffusion

Decision: accept (poster)
Reviewer scores: 6, 5, 8, 6

Reviews and discussion:
<thread>
creation date:
1730139919301

thread_type:
official_review

<thread_object>
creator: reviewer_kepg

<content>
summary:
The paper presents a perspective on studying likelihoods for diffusion models. They introduce a stochastic way to keep track of the likelihood. They introduce a likelihood maximizing ODE that maximizes the likelihood of a path for any given starting condition. They attempt to argue that they can use this ODE to demonstrate that the high likelihood region of diffusion models results in cartoon like images.

strengths:
- The paper is well written and the use of good figures and colors in equations really helps the presentation
- They present a stochastic way to keep track of the likelihood and a likelihood maximizing ODE

weaknesses:
- Up to section 4.2 everything was very reasonable and nicely introduced, however the use of the ODE in remark 1 for sampling is a big mistake. Real world data is multimodal, for instance CIFAR 10 has at least 10 modes. To use the assumption that the data is gaussian in order to use remark 1 results in incorrect results. Specifically this ODE doesn't sample from the correct target distribution, and therefore we observe those cartoon like images. 
- Additionally when comparing the likelihood of images sampled normally or from the HP-ODE the likelihoods were measured using different methods. In section 4.3 it is explained how they compute the likelihood of $p(y_0|x_t)$. Empirically they show that it results in higher likelihood than $p(x_0 |x_t)$, however since these likelihoods are being evaluated in different ways this comparison doesn't make sense. If done correctly the sample $y_0$ would be evaluated using the same way as $x_0$, so we would evaluate the likelihood of the **same distribution**
- The paper has a limited amount of quantitative evidence
- There is no explanation on the hyperparameters that were used during sampling

questions:
Please see weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730470783980

thread_type:
official_review

<thread_object>
creator: reviewer_nsfn

<content>
summary:
The authors offer a theoretical framework for estimating the log probability of samples that follow SDE in diffusion models. They develop novel forward and reverse augmented dynamics that estimates log probability not only for the case where $\nabla \log p_{t}(x)$ is known precisely but for the case where the score function is known. Then, the authors provide novel upper and lower bounds on the bias of the proposed estimator and analyze them on the experimental data. Lastly, the authors apply their proposed theory to analyze the diffusion probability landscape. They propose a simple yet effective way to generate samples with the highest log probability and show that such samples are unrealistic and blurry.

strengths:
* The authors develop a novel theory of augmented SDEs. They provided a clear and detailed derivation and coupled it with the bias estimation 
* Landscape analysis gives valuable insight into the structure of high-probability samples and provides a theoretical justification for the known fact that distorted images tend to have a higher likelihood 
* The paper is well-structured and easy to read

weaknesses:
1. It isn't clear whether analysis from Section 5 can lead to the creation of better stochastic samplers or improve the quality of image generation. Overall practical implications of this work are quite poor 
2. There is no intuition behind observations from Figure 4. More precisely, what does it mean that the model optimized for sample quality yields a smaller difference between $p_{0}^{ODE}$ and $p_{0}^{SDE}$?
3. Experiments were conducted on rather small and outdated diffusion models. It would greatly improve the scope of the work if the experiments were performed on the frontier models

questions:
1. Have the authors thought about how the proposed theory can be used to improve the quality of image generation?
2. Can we use the proposed estimation for the log probability to evaluate model quality the same way as [1] (Table 2)? Will there be any difference between SDE and ODE estimation? Which one is better to compare different models?

[1] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020c.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731054537912

thread_type:
official_review

<thread_object>
creator: reviewer_p2yw

<content>
summary:
This paper investigates the high-density regions of diffusion models, discovering that samples in these regions often appear as unrealistic cartoon-like or blurry images, despite the absence of such images in the training data. The authors propose a novel framework based on augmented stochastic differential equations (SDEs) to estimate the likelihoods of generated samples. This approach enables efficient, high-likelihood sampling without additional computational cost. The authors also introduce a high-probability sampling method that consistently yields higher-likelihood images than traditional sampling techniques, while their empirical analysis shows that images with less detail (e.g., blurred images) tend to achieve higher likelihood scores. The paper contributes to a better understanding of the diffusion model probability landscape and the relationship between likelihood and image quality.

strengths:
The paper introduces a novel framework for estimating likelihoods within diffusion models using augmented stochastic differential equations (SDEs) and high-probability samplers. This is an important advancement because it allows the exploration of high-likelihood regions without increasing computational costs. By deriving density estimates through augmented SDEs, the authors provide a theoretically efficient approach to analyzing model outputs across noise levels, setting a foundation for future studies in likelihood-based generative modeling. The findings reveal that high-likelihood samples often resemble cartoonish or blurry images—even though such images are not present in the training data.

weaknesses:
1. Although the paper sheds light on high-density regions and likelihood estimation, it lacks a clear discussion on how these findings could practically inform the design or improvement of diffusion models in applied settings. Without recommendations on balancing high-likelihood sampling with image quality, it’s challenging to draw valuable insights from the findings, particularly for practitioners focused on real-world applications.

2. The paper describes the emergence of cartoon-like images in high-likelihood samples but provides limited exploration into why or how this phenomenon occurs. 

3. The experiments are primarily conducted on well-known datasets like CIFAR-10 and FFHQ-256. Expanding the study to other types of diffusion models or additional data domains would strengthen the generalizability of the findings. Additionally, clarifying any specific architectural limitations of the proposed high-probability sampler could benefit those aiming to extend the approach.

4. While the paper emphasizes the likelihood of generated samples, it does not address diversity within these high-likelihood outputs. The authors mention blurry and cartoon-like images, but they do not provide an assessment of diversity or how the high-likelihood sampling might affect the overall variance in generated outputs. It’s unclear if the method leads to mode collapse or reduces the richness of the model’s output space.

questions:
1. Does the high-likelihood sampling technique impact the diversity of generated samples, potentially leading to mode collapse or other reductions in output variability? 

2. For applications that prioritize both high likelihood and visual quality, what adjustments to the high-probability sampler could mitigate the production of low-quality images?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731078683333

thread_type:
official_review

<thread_object>
creator: reviewer_nhxg

<content>
summary:
This paper introduces new augmented SDE for likelihood estimation in SDE sampling of diffusion models. Moreover, a novel theoretical mode-tracking approach is proposed in order to locate the exact mode of the generative distribution and introduce a high-probability sampler capable of generating samples with higher likelihood than all the other samples, under theoretically proved guarantees. 
Finally, the main findings of the proposes analysis lies in the discovery of Cartoon-like images that lies in high-likelihood regions, even if the model has never been trained with images of the same style.

strengths:
- The paper is well written and it develops new theoretical insights, especially the augmented SDE for tracking likelihood evolution are innovative and pratical since they don't introduce any additional computational cost.

- The proposed high-probability sampler is a nice tool to generate high-likelihood samples that are not discoverable by traditional sampling techniques

- The paper analize the diffusion probability landscape finding analyzing the different images in high-likelihood regions, experimentally demonstrating that the proposed theoretical tools can be used to perform analyisis of this landscape.

weaknesses:
- The proposed mode-tracking approach has a very high computational cost and this is not clearly discussed. The discussion on how this mode-tracking approach scale and its computational limitations should be discussed and quantified

- The high-likelihood samples discovered by the proposed analysis does not seem to have a real practical advantage, being cartoon drawings or blurry images. It is not discussed how these insight can be leveraged to improve sample generation strategies or how these high-density samples can be practically useful.

- The limitations of the work are not discussed at all, a limitations section covering all the main potential limitations should be added.

- The work does not report any implementation details, code and implementation are not submitted in the supplementary materials. It would be important to openly release the contribution upon acceptance to ensure reproducibility and improve transparency.

- The analysis of high-density regions is done on small and simple diffusion models trained on restricted datasets with limited variability. Moreover the selected models and study is done on uncoditional sampling without any guidance from text. In my opinion it would be valuable to explore real-world and more complex diffusion models (such as SD, Flux etc), especially focusing on the impact of text.

questions:
- The related work does not take into consideration a very relevant paper "Null-text Guidance in Diffusion Models is
Secretly a Cartoon-style Creator", Zhao et al which is not discussed. In particular it would be relevant to highlight some connection and insights with this previous related work. 

- See weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1732029240081

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response (1/2)

comment:
We thank the reviewers for their insightful comments and suggestions to improve this work, and the area, program, and general chairs for their great service to the community.

**Positive Aspects.** The Reviewers variously acknowledged many strengths of the paper, including clear structure and good writing (NhXG, nSFN, KEPG); novel theoretical insights (NhXG, p2yw, nSFN, KEPG); analysis of the probability landscape (NhXG, p2yw, nSFN).

**Concerns and Questions.** The Reviewers also provided constructive feedback, especially regarding the practicality of the high-density sampler and the lack of evaluation on more sophisticated models. Below we summarize the main concerns.

### How can this study lead to improved sampling strategies?
In our paper, we developed an efficient method for generating high-density samples. Reviewers (NhXG, p2yw, nSFN) asked how our findings can be leveraged to design improved samplers. Indeed, the high-density sampler does not produce high-quality images. Its purpose was to study high-density regions and we concluded that such regions do not contain high-quality samples. We have also provided additional intuition for why it happens (Appendix P in the revised manuscript).

Our study of high-density regions was inspired by [1], where the authors attributed the success of diffusion models to their ability to avoid low-density regions. Our analysis in section 5 shows that for high-quality image generation, the highest densities should also be avoided. Therefore, in the quest to improve perceptual quality, one might need to target "moderate" densities and perhaps even on the lower end (given our new discussion section in Appendix P, where we argue that negative loglikelihood measures the amount of detail in an image). We leave this exciting research direction for future work.

**We also take this opportunity to emphasize that the high-density sampler is not the sole contribution of this work. The likelihood estimation tools we introduced in sections 2 and 3 have other potential applications.** As we mention in line 157, one can e.g. use stochastic sampling for importance sampling as we now show that it is possible to generate both a sample and its likelihood estimate for no extra cost.

### Lack of analysis of more sophisticated models.
Reviewers (NhXG, p2yw, nSFN) suggested that it would improve the quality of the study if we included more sophisticated models in our analysis. We would like to point out that in our study we experimented with diffusion models up to 256x256 resolution, which we believe are relatively large.

However, based on this suggestion, we included an experiment with Stable Diffusion v2.1 [2], which is a text-conditioned latent diffusion model generating images in 768x768 resolution. Interestingly, even though this is a text-guided diffusion model operating in the latent space, we observed similar behavior to the diffusion SDEs in the pixel space. Specifically, high-density samples decode to images that are blurry or exhibit cartoon-like features. Please see Appendix O in the revised manuscript for details.

### Updated manuscript.
We uploaded a revised version of the manuscript with addressed concerns from all Reviewers and below we summarize the changes made.

1. **Change in terminology: "high-probability" -> "high-density".** to avoid any confusion and emphasize that we refer to sets of points, which have high probability density.
2. **Model hyperparameters** - We added Appendix M: "CIFAR models hyperparameters", where we detail the hyperparameters and the training procedure used for CIFAR models in our experiments.
3. **Computational cost of mode-tracking** - We added K: "Cost of mode-tracking", where we discuss in detail the cost of evaluating the exact mode-tracking ODE introduced in Theorem 5.
    * Specifically, we show that evaluating a single step in solving the mode-tracking ODE scales worse than linearly with the dimension of the input $D$, which for CIFAR10 amounts to computation equivalent to over 6000 score function evaluations and to over 400000 score function evaluations for 256x256 data. In comparison, PF-ODE and HD-ODE both require a single score function evaluation per step.
4. **More details on the experiments** - We provide in the text details regarding sample sizes used in the generation of Figures 7, 8, 9, and 10.
5. **A quantitative analysis of HD sampling** - We added Appendix N: "Quantitative analysis of likelihoods of samples generated with algorithm 1", where provide a table containing $\mathbb{E}[-\log p_0(x_0)]$ for all baselines and different sampling strategies and different values of the threshold parameter $t$.
    * We reinforce our claims that the HD sampler indeed generates points with higher likelihoods than regular samples across different models.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a mode-tracking method to sample data from high-density regions from a diffusion model. To this end, a likelihood estimation method is proposed and theoretical guarantees are given. Empirically, this paper found that the high likelihood examples are more cartoon-like drawings or blurry images. The reviewers agree that the paper has novel technical and theoretical contributions, but the application of the proposed method is unclear. The authors have partially addressed this concern in the rebuttal. Overall, I think it is a solid paper and recommend accepting it to ICLR.