Ambient Diffusion Omni: Training Good Models with Bad Data
Authors: Giannis Daras
Adrian Rodriguez-Munoz
Adam Klivans
Antonio Torralba
Constantinos Costis Daskalakis
Venue: NeurIPS 2025 spotlight ()
Keywords: ambient diffusion
diffusion models
corrupted data
generative AI
Primary area: deep_learning
TL;DR: We improve the quality of generative models by using low-quality, corrupted, and out-of-distribution data

Abstract: We show how to use low-quality, synthetic, and out-of-distribution images to improve the quality of a diffusion model. Typically, diffusion models are trained on curated datasets that emerge from highly filtered data pools from the Web and other sources. We show that there is immense value in the lower-quality images that are often discarded. We present Ambient Diffusion Omni, a simple, principled framework to train diffusion models that can extract signal from arbitrarily images during training. Our framework exploits two properties of natural images -- spectral power law decay and locality. We first validate our framework by successfully training diffusion models with images synthetically corrupted by Gaussian blur, JPEG compression, and motion blur. We use our framework to achieve state-of-the-art ImageNet FID and we show significant improvements in both image quality and diversity for text-to-image generative modeling. The core insight is that noise dampens the initial skew between the desired high-quality distribution and the mixed distribution we actually observe. We provide rigorous theoretical justification for our approach by analyzing the trade-off between learning from biased data versus limited unbiased data across diffusion times.

Ambient Diffusion Training Good Models with Bad Data
CO mni:
Giannis Daras Massachusetts Institute of Technology gdaras@mi it.
Adrian Rodriguez-Munoz
Massachusetts Institute of Technology
adri sarianmmmmit... edu
Adam Klivans
Antonio Torralba
The University of Texas Austin klivans@utexas.eem
Massachusetts Institute of Technology torralba mit.edu
Constantinos Daskalakis
Massachusetts Institute of Technology costis@csail. .mmt....... edu
Abstract
We show how use low-quality, synthetic, and diittii-ditiiittionnn images improve the quality of diffusion model. Typically, diffusion models are trained on curated datasets that emerge from highly filtered data pools from the Web and other sources. We show that there is immense value in the lower-quality images that are often discarded. We present Ambient Diffusion Omni, simple, principled framework to train diffusion models that can extract signal from all available images during training. Our framework exploits two properties natural images spectral power law decay and locality. We first validate framework by successfully training diffusion models with images synthetically corrupted by Gaussian blur, JPEG compression, and motion blur. We then of-the-art ImageNet FID and we show significant improvements in both image quality and diversity for text-to-image generative modeling. The core insight is that noise dampens the initial skew between the desired high-quality distribution and the mixed distribution we actually observe. We provide rigorous theoretical justification for our approach by analyzing the trade-off between learning from biased data versus limited unbiased data across diffusion times.
framework to achieve state-
Introduction
Large-scale, high-quality training datasets have been primary driver of recent progress in generative modeling. These datasets are typically assembled by filtering massive collections images sourced from the web proprietary databases [25, 44. 53, 58, 59]. The filtering process which determines which data retained crucial to the quality of the resulting models [13, 25, 27, 33]. However, filtering strategies often heuristic and inefficient, discarding large amounts of data [13, 25, 44, 51]. We demonstrate that the data typically rejected as low-quality holds significant, underutilized value.
Extracting meaningful information from degraded data requires algorithms that explicitly model the degradation process. In generative modeling, there is growing interest in approaches that learn to generate directly from degraded inputs [1, 2, 5, 7, 11, 14, 15, 17, 18, 31, 40, 46, 47, 48, 52, 55,
"Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

64. 71, 72]. A key limitation of existing methods is their reliance on knowing the exact form of the degradation. real-world scenarios, image degradations such as motion blur, sensor artifacts, poor lighting, and low resolution- are often complex and lack a well-defined analytical description, making this assumption unrealistic. Even within the same dataset, from ImageNet to internet scale text-to-image datasets, there are samples of heterogeneous qualities [28], as shown in Figures 4, 25, 28, 26. Given access to this mixed-bag of datapoints, we would like to sample from a tilted continuous measure of high-quality images, without sacrificing the diversity present in the training points.
ol's 0orse, chhorati dhed"
semrost intrra pir"
card in Artincioo
Boseline
Boseline
Ambicut-o
Ambient-o
a) Text-to-image results
b) ImageNet results
Figure 1: Effect of using Ambient-o for (a) training text-to-image model IMicro-Diffusion [54]) and (b) caasscconiiionall model for ImageNet (EDM-2 [36]). All generations are initialized with the noise. The baseline models trained using all the data equally. Ambient-o changes the way the data used during the diffusion process based on its quality. This leads to significant visual improvements without sacrificing diversity, as would happen with a filtering approach (see Fig. 6).
The training objective of diffusion models naturally decomposes sampling from a target distribution into sequence of supervised learning tasks [9, 10, 16, 19, 30, 61, 62]. Due to the power-law structure of natural image spectra [65], high diffusion times focus on generating globally coherent, semantically meaningful content [22], while low diffusion times emphasize learning high-frequency details.
Our first key theoretical insight that low-quality samples can still be valuable for training in the high-noise regime. As noise increases, the diffusion process contracts distributional differences (Theorem 4.2), reducing the mismatch between the high-quality target distribution and the available mixed -ixee--uulity data. At the same time, incorporating low-quality data increases the sample size, reducing the variance of the learned estimator. Our analysis formalizes this bias-variance trade-off and motivates principled algorithm for training denoisers at high diffusion times using noisy data.
For low diffusion times, our algorithm leverages second key property of natural images: locality. We show direct relationship between diffusion time and the optimal receptive field size for denoising. consequence of this result that high-frequency details can be learned from out-of-distribution or synthetic images, as long as the marginal distributions of the crops match those of the target data.
We introduce Ambient Diffusion Omni (Ambient-o), simple and principled framework for train- ing diffusion models using arbitrarily corrupted and out-of-distribution data. Rather than filtering samples based on binary 'good' 'bad' labels, Ambient-o retains all data and modulates the training process according to each sample's 'sl utility. This enables the model to generate diverse outputs without compromising image quality. Empirically, Ambient-o advances the state of the art in unconditional generation on ImageNet and enhances diversity in text-conditional genera- tion without sacrificing fidelity. Theoretically, it achieves improved bounds for distribution learn- ing by optimally balancing the biai-variance trade-off: low-quality samples introduce bias, but their inclusion reduces variance through increased sample size. We release our code and models: https://github.c wtttcthhuu.oomoeeeeee wtw.g//iiiiiaaam/mmmitttoooooomo

---

2 Background and Related Work
Diffusion Modeling. Diffusion models transform the problem of sampling from po into the problem
of learning denoisers for smoothed versions of Po defined as Pt = Po @ (N (t)T). We typically denote with Xo Po the R. distributed according to the distribution of interest and X, Xo + 2t the R.V. V. distributed according to Pt- The target is to estimate the set of optimal l2 denoisers, i.e., the set of the conditional expectations: [IE[Xo|X, - -D Typically, this can be achieved through supervised learning by minimizing the following loss (or a re-parametrriztion of it):
J(0) = Exetr (0.7) B [1o 1) =
(2.1)
that is optimized over function family H. (he 0 E Q} parametrized by network parameters O. For sufficiently expressive families, the minimizer is indeed: he- (x, t) = B[Xo X; = z].
Learning from noisy data. The diffusion modeling framework described above assumes access to samples from the distribution of interest Po. An interesting variation of this problem to sample from Po given access to samples from a tilted measure Po and a known degradation model. In Ambient Diffusion [18], the goal is to sample from po given pairs (Azio A) for matrix A R , < that is distributed according to a known density p( A). The techniques in this work were later generalized to accommodate additive Gaussian Noise [1, 15, 17] the measurements. More recently there have been efforts to further broaden the family of degradation models considered through Expectation- Snspctation-MMxxiizzttinn approaches that involve multiple training runs [5, 52].
learn
Recent work from [17] has shown that, at least for the Gaussian corruption model, leveraging the low-quality data can tremendously increase the performance of the trained generative models. In particular, the authors consider the setting where we have access to few samples from po, let's
denote them Do Do
(4) Ji=i where (t) Ni and many samples from Pt,, let's denote them Dt,, {
Pen = Po @ (0, o² (f,, )I) a smoothed version of po a known noise level t. The clean samples are used to learn denoisers for all noise levels t E j0, T) while the noisy samples are used to learn denoisers only for t 2 tx using the training objective:
N2 E
- x 1
Jambiiett (0) = EEEL((
> 1=1
+) ()
(2.2)
with (xf) (e())) Note that the objective of equation 2.2 only requires samples from pt. (+)
(instead of po) and can be used to train for all times / > L,. This algorithm uses N; N2 datapoints to learn denoisers for f. >> i, and only Nj datapoints to learn denoisers for S f,,- The authors show that even for N1 << N2. the model performs similarly to the setting of training with (N1 N2) clean datapoints. The main limitation of this method and its related works is that the degradation process needs to be known. However, in many applications, we have data from heterogeneous sources and various qualities, but there is no analytic form or any prior on the corruption model.
Data filtering. One of the most crude, but widely used, approaches for dealing with heterogeneous data sources is to remove the low-quality data and train only the high-quality subset [23, 25, 44). While this yields better results than naively training on the entire distribution, it leads to decrease in diversity and relies on heuristics for optimizing the filtering. An alternative strategy is to train on the entire distribution and then fine-tune on high-quality data [13, 54]. This approach better trades the quality-diversity trade-off' but still incurs a loss of diversity and is hard to calibrate.
Training with synthetic data. Recent works have shown that synthetic data improve the generative capabilities of diffusion models when mixed properly with real data from the target distribution [3, 4, 24]. In this work, we show that it helps significantly to view synthetic data as corrupted versions of the samples from the real distribution and incorporate this perspective into the training objective.
3 Method
We propose a new framework that extends beyond [17] to enable training generative models directly from arbitrarily corrupted and -uitof-distributtion data, without requiring prior knowledge of the degradation process. We begin by formalizing the setting of interest.
3

---

image
Merging point
Distributionns merge laurry
Cless
ange
Wtary
0.5
0.3
Probabihtt Soore Threshold (0.45)
0.0
Noise stádev
Figure 2: A time-dependent classifier trained to distinguish noisy clean and blurry images (blur kernel standard deviation оB = 0.6). At low noise the classifier is able to perfectly identify the blurry images, and outputs probability close O. As the noise increases and the information in the image is destroyed, the clean and blurry distributions converge and the classifier outputs prediction close 0.5. The red line plots the threshold (selected at T 0.45), which is crossed at o 1.64.
Problem Setting. We are given a dataset = (w) | consisting of datapoints. Fach point in D is drawn from a mixture distribution Po. which mixes Po (target distribution) and an alternative distribution qo that may contain various forms of degradation or out-of-distribution content. We assume access to two labeled subsets, Sc, Sg. where points in SG are known come from the clean distribution Po. and points SB from the corrupted distribution go. While this assumption simplifies the initial exposition,
relax in Section G.I. We focus on the practically relevant regime where |Sa| [D], ie.. access to high-quality data severely limited. The objective to (approximately) sample from the clean distribution Po. leveraging both clean and corrupted samples.
We now describe how degraded and out-of-distribution samples can be effectively leveraged during training in both the high-noise and low-noise regimes of the diffusion process.
3.1 Learning in the high-noise regime (leveraging low-quality data)
Addition of gaussian noise contracts distribution distances. The first key idea of our method is that, at high diffusion times the noised target distribution pt and the noised corrupted distribution p. become increasingly similar (Theorem 4.2), effectively attenuating the discrepancy introduced by corruption. This effect is illustrated in Figure (top), where we compare clean image and its degraded counterpart (in this case, corrupted by Gaussian blur). As the diffusion time f increases, the noised versions of both samples become visually indistinguishable. Consequently, samples from Po can be leveraged to learn (the score of) pt, for / > imin We formalize this intuition in Section 4, and we also quantify that for large there are statistical efficiency benefits for using large sample from Po versus a small sample from Po
Heuristic selection of the noise level. From the discussion so far, follows that to use samples from Po. we need to assign them to noise level emin One can select this noise level empirically, iec. we can ablate this parameter by training different models and selecting the one that maximizes the generative performance. However, this approach requires multiple trainings, which can be costly, Instead, we can find the desired noise level principled way as detailed below.
Training classifier under additive Gaussian noise. To identify the appropriate noise level, we train time-conditional classifier distinguish between the noised distributions p. and q. across various diffusion times. We use a single neural network egoise ()) that is conditioned on the diffusion time following the approach of time-aware classifiers used in classifier guidance [21]. The classifier is trained using labeled samples from SG (clean) and SB (corrupted) via the following objective:
Juoise (0) = Σ
Ezz,zo [-
[-log log cB mnise(Xr, ()] + Σ Ey vo [- logg(1 cooii (v+, t))] BoESp
poise
(3.1)
Annotation. Once the classifier trained, we use it to determine the minimal level of noise that must be added to the low-quality distribution go so that it closely approximates smoothed version of the

---

high-quality distribution Po. Formally, we compute:
imin = inf L [0, 12
(Mt, 11] >
(3.2)
SB
WoESn
for r = - - and for some > 0. Subsequently, we form the annotated dataset D snnot {(u)) ( +0 ( ( i 0)|zo E Sc). where the random variables (6) are standard normals. This procedure means that we use samples from "D for diffusion times 2 1m that is safe to use them, i.e. only when the distributions have approximately merged. In fact, the optimal classifier assigns time in that corresponds to the first time for which drv (Pt, qt) S.
Sample dependent annotation. One poten- tial issue with the aforementioned annotation approach is that all the samples in D are treated equally. But, as we noted, the points in D could be drawn from a distribution Po that mixes Po and go- In this case, all the samples in D that came from the Pu component, will still get high annotation time, leading to information loss. Instead, we can opt-in for a sample-wise
2-4 and Ayiiiarr
by dingen oneyeee
IRQ
Ganoian Blar
JIEG
MaGolllar
annotation scheme, where each sample wb gets assigned time t.min based on: tmmin inf E [0, T] E (uge) [cgolse (we, t)] r} for r 0.5 and for some
Masking
Figure 3: Visual summary of our method. Cor- rupted images become indistinguishable from the high quality ones after a certain noise level. These noisy ver- sions are actually useful for learning; traditional filtering approaches discard them, but Ambient Omni uses them.
From arbitrary corruption to additive Gaus- sian noise. The aesre-de approach re- duces problem of learning from data with ar- bitrary corruption to the setting of learning from data corrupted with additive Gaussian noise.
The price we pay for this reduction is the information loss due to the extra noise we add to the samples during the annotation stage. We can now extend the objective 2.2 to train diffusion model.
Suppose our annotated dataset is comprised of samples { {ffffmm (mii) Then our objective becomes:
[latt. th + (1 att, (mit (2xi x
112
Jambient (0) ErEZ(O,
Σ
E 22 suiin
gunin
dimminc st
where att,, (min)
Learning something from nothing? The proposed framework comes with limitations worth considering. First, unless the diffusion noise level tends infinity, the distributions pt and qi never fully converge-there is always a bias when treating samples from ge as if they were from
Moreover, the method is particularly well-suited to certain types of corruptions but is less effective for others. Because the addition Gaussian noise suppresses high-frequency components due the spectral power law of natural images-our approach is most effective for corruptions that primarily degrade high frequencies (e.g.. blur). In contrast, degradations that affect low-frequency content-such as color shifts, contrast reduction, fog-like occlusions-aa limitation is illustrated Figure 3: masked images, for example, require significantly more noise to become usable compared high-frequency corruptions like blur. In the extreme, the method reduces to filtering approach, as infinite noise nullifies all information in the corrupted samples.
more challenging. This
3.2 Learning in the low-noise regime (synthetic and
data)
So far, our algorithm implicitly results varying amounts of training data across diffusion noise levels. At high noise, the model can leverage abundant low-quality data, whereas at low noise levels,
5

---

it must rely solely on the limited set of high-quality samples. We now extend the algorithm to enable the use of synthetic and -utt---distrbbution data for learning denoisers low-noise diffusion times.
To achieve this, we leverage another fundamental property of natural images: locality. At low diffusion times, the denoising task can be solved using only a small local region of the image, without requiring full spatial context. We validate this hypothesis experimentally in the Experiments Section (Figures 15, 16, 17, 18), where we show that there is mapping between diffusion time and the crop size needed to perform the denoising optimally at this diffusion time. Intuitively, the higher the noise, the more context is required to accurately reconstruct the image. Conversely, for lower noise, the local information within a small neighborhood suffices to achieve effective denoising. We use crop(t) to denote the minimal crop size needed to perform optimal denoising at time t. If there are two distributions Po and Po that agree on their marginals (i.e. crops), they can be used interchangeably for low-diffusion times. Note that the distributions don't have agree globally, they only have to agree on a local (patch) level. Formally, let A(t) be random patch selector of size crop(t). Let also Po.Po Po two distributions that satisfy: #po measure" of po under A(t). Then, the cropped portions of the tilted distributions provide equivalent information to the original crops for denoising.
#fo where A(t) #Po denotes the pushforward
Training a crops classifier. Note that the condition above can be trivially satisfied if A(t) masks all the pixels or even if A(t) just selects a single pixel. We are interested finding what is the maximum crop size for which this condition is approximately true. Once again, we can use a classifier to solve this task. The input to the classifier, C crops is crop of an image that either arises from Po or Po. and the classifier needs to classify between these two cases.
Annotation and training using the trained classifier. Once the classifier is trained, we are now interested in finding the biggest crop size for which the distributions po. po cannot be confidently distinguished. Formally,
max R
sup
LE [OTT):
Co
cTOPS (A(1)()))
>
(3.3)
SB
BoESa
for = 0.5 and for some small > 0º. For times ( < mmax the out-of- images from Po can be used with the regular diffusion objective as images from Po. as for these times the denoiser only looks at crops and at the crop level the distributions have converged.
The donut paradox. Each sample can be used for 2 emin and for S E.max but not for (1max, rqinn.. We call this the donut paradox as there is a hole in the middle of the diffusion trajectory for which we have fewer available data. These times do not have enough noise for the distributions to merge globally, but also the required receptive field for denoising is big enough so that there are differences on a crop level. We show an example of this effect Figure 14.
Figure 4: Results using CLIP to obtain the high-quality and the low-quality sets of ImageNet.
4 Theory
We study the 1-d case, but all our claims easily extend to any dimension. We compare two algorithms:
Algorithm 1. Algorithm 1 trains a diffusion model using access to n, samples from target density Po. assumed to be supported in [0. 1] and be -i-Lipschitz.
Algorithm 2. Algorithm 2 trains a diffusion model using access to n1 + n2 samples from a density Oo that is mixture of the a target density Po and another density go assumed to be supported in [0, 1
and be -z-Lipschitz: Ωo = miting Po + mitnng 40-
mi
n2
Given measure spaces (X1, En) and (X2, M2), a measurable function X1 and a probability X2,
measure p: Σ, + [0, oo), the pushforward measure o fH is defined as (B) := (B)) VB E E22.
We subtract allow for approximate mixing of the distributions and hence smaller annotation times.

---

We want to compare how well these algorithms estimate the distribution := @ &N(0, We use p.
to denote the estimates obtained for pt by Algorithms and respectively.
Diffusion modeling is Gaussian kernel density estimation. We start by making connection between the optimal solution to the diffusion modeling objective and kernel density estimation. Given finite dataset (W()) Jimi the optimal solution the diffusion modeling objective should match the empirical density at time i, which is:
n
W
P((x)
Φ
(4.1)
not
where 6(u) 1 e -2/22 the Gaussian kernel. We observe that equation 4.1 is identical to Gaussian kernel density estimate, given samples {/(") 4.
We establish the following result for Gaussian kernel density estimation.
Theorem 4.1 (Gaussian Kernel Density Estimation). Let (W()) be set of independent samples from a X-Lipschitz density p. Let P be the empirical density. Pa == p @ (0, o²) and Po = 6 @ (0, d²). Then, with probability at least 1 with respect the sample randomness,
log n + log(1 V X) 1og 2/5
(po Po) x
(4.2)
o2n
82n
The proof of this result is given in the Appendix.
Comparing the performance of Algorithms and 2. Applying Theorem 4.1 directly to the po density, we immediately get that the estimate pfa)(() obtained by Algorithm satisfies:
log log n1 - log(1 V A1) + log 2/6
drv(pr, Pt
+
(4.3)
ob
ofn1
Let us now what get by applying Theorem 4.1 Algorithm 2, which samples from the tilted distribution Po. Since this distribution is (nitmn A1 + n2 )2) Lipschiitz we get that:
log(ni + n2) + log(1 V X1 a2 mitm d2) + log 2/5
drv(Pt. ((2)) S (ni + n2) + of + m2)
o [m1 + 12)
where p. = 6o @ O0)
Further, have that: drv (Pt. p(22) S drv pc) (f. We already have bound for the second term. To bound the first term, we prove the following theorem.
Theorem 4.2 (Distance contraction under noise). Consider distributions P and Q supported on subset of Rdd with diameter D. Then
drv (10 2 S ddv((,.Q) D
20
Applying this theorem we get that: dTv Pc) S dTv - L "ル 20.
drv qo), where
20,
for the second inequality we used that drv (Po. Pa) S nitme drv dTv(po: 9o)
B2
Putting everything together, Algorithm (2) achieves an estimation error:
(rv(pt.p(2) E
log(m 22) + log(1 B1 81
82
Ax) log 2/6
of(n, + nz) +1
mi
drv (pa-90).
(mg n2)
of (m, + n2)
an + n2)
*This connection has been observed in prior works too, e,g., [8. 34).

---

Comparing this with the bound obtained in Equation 4.3, we see that if n2 is sufficiently larger than n, or if X2 S A₁, there is tmin such that for any t 2 tmmi the upper-bound obtained by Algorithm 2 is better than the upper-bound obtained by Algorithm 1. That implies that for high-diffusion times, using biased data might be helpful for learning, as the bias term (final term) decays with the amount of noise. Going back to equation 4, note that the switching point & 2 mnin depends on the distance drv (fr. that decays as shown in Theorem 4.2. Once this distance becomes small enough, our computations above suggest that we benefit from biased data. The classifier of Section 3.1, if optimal, exactly tracks the distance drv (Pr, Pt) and, as a result, tracks the switching point.
5 Experiments
Controlled experiments to show utility from low-quality data. To verify our method, we first do synthetic experiments on artificially corrupted data. We use EDM [35] as our baseline, and we train networks on CIFAR-10 and FFHQ For the first experiments, we only use the high-noise part of our Ambient-o method (Section 3.1). We underline that for all of our experiments, we only change the way we the data, and we keep all the optimization and network hyperparameters as is. We compare against using all the data as equal (despite the corruption) and the filtering strategy of only training on the clean samples. For evaluation, we measure FID [29] with respect the full uncorrupted dataset (which is not available during training). For the blurring experiments, we use a
Table I: In a controlled experiment with restricted access only to 10% of the clean dataset, our method of Ambient-o uses corrupted and oft-of--istribuue data to improve performance.
(a) Gaussian blurred data at different levels.
(b) Additional out-of-distribution data.
Method
Parameters Values
t.
FID
Source Data
Additional Datu
Method
mmax
FID
Only Clean (10%)
8.79
None
Cais
Fixod o
0.2
11.11
45.32
Cats
Fixed
0.1
9.85
All data
0.8
0
28.26
Dogs (10%)
Cats
Fixed
0.05
10.66
0.6
11.42
Cats
Fixed
0.025
12.07
0.4
2.47
Cats
Classitier
0.09
8.92
1.0
2.84
6,16
Procedural
Classilee
0,042
10.98
Ambient-o
0.8
0.6
0.22
1.98
1.38
2.44
6.00
5.34
Cats (10%)
Wildlife
None
Dogs
Classitter
Classifier
0.13
0.08
4.89
5.20
Gaussian kernel with standard deviation dB = 0.6, 0.8, 1.0, and we corrupt 90% of the data, We show some corrupted images in Figure 9a. To perform the annotations for our method, we train blurry image vs clean image classifier under noise, as explained in Section 1. For the experiments in the main paper, we use balanced dataset for the training of the classifier. We ablate the effect of having fewer training samples in Appendix Section F, where we show that reducing the number of clean samples available for classifier training leads small drop in performance. Once equipped with the trained classifier, each sample is annotated on its own based on the amount of noise that needed to confuse the classifier (sample-dependent annotation). We present results in Table la. As shown, for all corruption strengths, Ambient Omni, significantly outperforms the two baseline methods. In the one to the last column of Table 1a, we further show the average annotation of the classifier. As expected, the average assigned noise level increases as the corruption intensifies.
Ablations. We ablate the choice of using fixed vs cample-adaptive annotations in Table 12. We find that the latter performs better, but both methods improve over the baselines. We present results with JPEG compression in Table 3, motion blur in Figure 10 and FFHQ in Table 4. We ablate the impact of the amount of training data and training iterations on the classifier in Section F.
images. We now want validate the method developed in Section 3.2 for leveraging crops from di-ttr-iiitiittti data. To start with, we want to find the mapping between diffusion times and the size of the receptive field required for an optimal denoising prediction. To do so, we take a pre-trained denoising diffusion model and measure the denoising loss at given location as increase the size of the context. We provide the corresponding plot in the Supplemental Figures 17, 15. The main finding is that while providing more context always leads to a decrease in the average loss, for sufficiently small noise levels, the loss nearly plateaus before the full image context is provided. That implies that the perfect denoiser for a given noise level only needs look at a localized part of the image.
Controlled experiments to show utility from
8

---

Equipped with the mapping between diffusion times and crop sizes, we now proceed fun experiment. We show that it is possible use images of cats to improve a generative model for dogs (!) and vice-versa. The cats here represent out-of-distribution data that can be used to improve the performance in the distribution of interest (in our toy example, dogs distribution). To perform this experiment, we train classifier that discriminates between cats and dog images by looking at crops of various sizes (Section 3.2). Figure 5 shows the predictions of 8 x 8 crops-classifier for image of cat, illustrating that there are a number of crops that are misclassified as crops from a dog image. We report results for this experiment in Table 1b and we observe improvements in FII) arising from using out-of-distribution data. Beyond natural images, we show that it is even possible to use procedurally generated data from Shaders [6] to (slightly) improve the performance. Figure 21 shows an example of such an image and the corresponding predictions of crops classifier. Table 1 contains more results and ablations between annotating all the out-of- distribution at a single noise level vs. sample-dependent annotations.
Figure 5: Patch level probabili- ties for dogness in a cat image.
Takeaway 1: It is possible use low-quality in-distribution images and high-quality out-of- distribution images to produce high-quality in-distribution images.
Corruptions of natural datasets ImageNet results. Up to this point, our corrupted data has been artificially constructed to study our method in a controlled setting. However, it turns out that even in real datasets such as ImageNet, there are images with significant degradations such as heavy blur, low lighting, and low contrast, and also images with fantastic detail, clear lightning, and sharp contrast. Here, the high-quality and the low-quality sets are not given and hence we have to estimate them. We opt to use the CLIP-IQA quality metric [66] to separate ImageNet into high- (top 10% CLIP-IQA) and low-quality (bottom 90% CLIP-IQA) sets. Figure 4 shows some of the top and bottom quality images according to our metric. Given the high-quality and low-quality sets, we are now back to the previous setting where we can use the developed Ambient-o methodology. We underline that there is a rich literature regarding guality-assesssment methods [49, 67, 68, 69].
We use Ambient-o to refer to our method that uses low-quality data at high diffusion times (Section 5) and Ambient-o+crops to refer to the extended version of our method that uses crops from potentially low-quality images low-diffusion times. Perhaps surprisingly, there are ImageNet images that have lower global quality but high- -uaaltyy crops that we can use. We present results in Table 2, where we show the best FID [29] and FDDINOV2 obtained by different methods. We show the highest and lowest quality crops of ImageNet according to CLIP, alongside the full images, in Figure 11.
Table 2: ImageNet results with and without classifier-free guidance.
Train FID
Test FID
Model Size
ImageNet-512
FID
FIDv2
FID
FIDv2
no CFG
w/ CFG
no CFG
w/ CFG
no CFG
w/ CFG
no CFG
w/ CFG
Mparams
EDM2-XS
3.57
2.91
103.39
79.94
3.77
3.68
115.16
93.86
125
63
Ambient-o-XS
3.59
2.89
107.26
79.56
3.69
3.58
115.02
92.96
125
63
EDM2-XXL
(1.93)
1.81
4288
33.09
2.38
2.73
56.42
46.22
1523
63
Ambient-o-XXL
1.99
1.87
43.38
33.34
2.81
2.68
56.40
46.02
1523
63
Ambient XXILectt
42,84
32.63
2.78
2.53
56.39
45.78
1523
As shown in the Table, our method leads to state-of-the-art FID scores, improving over the baseline EDM-2 [36] at both the low and high parameter count settings. The benefits are more pronounced when we measure test FID as our method memorizes significantly less due to the addition of noise during the annotation stage of our pipeline (Section 3.1). Beyond FID, we provide qualitative results in Figure (bottom) and Appendix Figures 12, 13. We further show that the quality of the generated images measured by CLIP increased compared to the baseline in Appendix Table 5. The observed improvements are proof that the ability to learn from data with heterogeneous qualities can be truly impactful for realistic settings beyond synthetic corruptions typically studied in prior work.
Takeaway 2: Real datasets contain heterogeneous samples. Ambient-o explicitly accounts for quality variability during training, leading to improved generation quality.
Text-to-image results. For our final set of experiments, we show how Ambient-o can used to improve the performance of sat-to-image diffusioo models. We use the code-base of MicroDiffusion

---

[54], as it is open-data and trainable with modest compute (= 2 days on 8-H100 GPUs). Sehwag et al. [54] use four main datasets to train their model: Conceptual Captions (12M) [56], Segment Anything (11M) [42], JourneyDB (4.2M) [63], and DiffusionDB (10.7M) [70]. Of these four, DiffusionDB is of significantly lower quality than the others as contains solely synthetic data from an outdated diffusion model. This presents an opportunity for the use of our method. Can we use this lower-quality data and improve the performance of the trained network?
We set min = 2 for all samples from DiffusionDB and min 0 for all other datasets and we train diffusion model with Ambient-o. We note that we did not ablate this hyperparameter and it quite likely that improved re- sults would be obtained by tuning it or by training high-quality vs low- quality data classifier for the annota- tion. Despite that, our trained model achieves remarkable FID of 10.61 in COCO, significantly improving the baseline FID of 12.37 (Table 8). We present qualitative results in Figure 1
(a) "the great battle middle earth, unreal engine, trending on artstation, masterpiece" "
(b) "an abominable snowman trapped in ice hy greg rutkowski"
Figure 6: Examples of mode collapse. Left: baseline model
finetuned on a high-quality subset. Right: Ambient-o using all the data. As shown, finetuning decreases output diversity. and GPT-4o evaluations DrawBench and PartiPrompt in Figure 7. Ambient-o and baseline
generations for different prompts can be found in Figure 1.
100
100
Beseline
Ambientoo
Baseline
Ambient-o
80
73
Percentage (%)
Percentage (%)
55
54
57
40
33
35
35
34
20
20
Abaract
Aninols
Artifarts
Fond & Beverage
Bustr antraatioss
indear Stonec
Outdber Scenes
Produre Plants
Reople
World Knowledge
Vehictes
Carors
Conticting
Coanting
DeLL- 6
Desoriptions
Misspelling
Pasttional
Rare Warms
Redatt
Rest
Gid at
Figure 7: Assessing image quality with GPT-4o on DrawBench (left) and PartiPrompts (right).
(a) Measuring fidelity and prompt alignment of
(b) Mcasuring performance on the GenEval henchmark.
generated images COCO dataset.
Objocts
Method FID-30K (J)
Clip 30- (J)
Clip scorec (1)
Mathod
Overall
Single
Two
Couming
Colers
Pooition
attilltion
Color
Baseline
12.37
10.07
0.345
Boscline
0.35
0.82
0.86
Ambient-o
10.61
9.40
0,348
Ambient-o
0.47
0.97
0.40
0.36
0.82
0,11
0.14
Figure 8: Quantitative benefits of Ambient-o on COCO [45] zero-shot generation and GenEval [26].
As an additional ablation, compared our method with the recipe of doing final fine-tuning the
highest-quality subset, as done in [13, 54]. Compared to this bascline, our method obtained slightly worse COCO FID (10.61 vs 10. (0.27) but obtained much greater diversity, seen visually in Figure 6 and quantitatively through > 13% increases in DINO Vendi Diversity on prompts from DiffDB (3.22 vs 3.65.). This corroborates our intuition that data filtration leads to decreased diversity. Ambient-o uses all the data but can strike a fine balance between high-quality and diverse generation.
Takeaway 3: Ambient-o treats synthetic data corrupted data. This leads to superior visual quality and increased diversity compared to only relying on real samples.
Conclusion
Is it possible to get good generators from bad data? Our framework leverages low-quality, synthetic, and nut-of-distribution samples. At time when the ever-growing data demands of GenAl are odds with the need for quality control, Ambient-o lights a path for both to be achieved simultancously.
10

---

7 Acknowledgements
This research has been supported by NSF Awards CCF- ONR grants 300014-25-1-2111 16. N00014-25-1 2--22966, a Simons Investigator Award, and the Simons Collaboration on the Theory of Algorithmic Fairness. The experiments were run on the Vista GPU Cluster through the Center for Generative AI (CGAI) and the Texas Advanced Computing Center (TACC) at UT Austin. Adrián Rodríguez-Muñoo is supported by the La Caixa Fellowship ACC//Q//U22/113300044)
References
[1] Asad Aali, Marius Arvinte, Sidharth Kumar, and Jonathan I Tamir. "Solving Inverse Prob- lems with Ccore-Based Generative Priors learned from Noisy Data". In: arXiv preprint arXiv:2305.011 (2023) (cit. pp. 3).
[2] Asad Aali, Giannis Daras, Brett Levac, Sidharth Kumar, Alex Dimakis, and Jon Tamir. "Ambi- ent Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Data". In: The Thirteenth International Conference on Learning Representations.
2025. URL: https: /tpppppeeeeeeeeeeee net/forumm id qew.eutXccMettte (cit. on p. 1).
[3] Sina Alemohammad, Josue Casco-Rodriguet Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G Baraniuk. "Self-consuming generative models go mad". In: arXiv preprint arXiv:2307.0188
(2023), p. 14 (cit. on p. 3).
[4] Sina Alemohammad, Ahmed Imtiaz Humayun, Shruti Agarwal, John Collomosse, and Richard Baraniuk. "Self-improving diffusion models with synthetic data". In: arXiv preprint arXiv:2408. 16333 (2024) (cit. on p. 3).
[5] Weimin Bai, Yifei Wang, Wenzheng Chen, and He Sun. "An Expectation- Maximizaaonniiiiiiiinn Algorithm for Training Clean Diffusion Models from Corrupted Observations". In: arXiv preprint arXiv: 2440700014 (2024) (cit. on pp. 1,3).
[6] Manel Baradad, Chun-Fu Chen, Jonas Wulff, Tongzhou Wang, Rogerio Feris, Antonio Torralba, and Phillip Isola. "Procedural Image Programs for Representation Learning". In: Advances in Neural Information Processing Systems. Ed. by Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho. 2022. URL: https: wwwnreviewwweee..oo.p.... net/ wormmoormmmem// wd= (cit. on p. 9).
[7] Ashish Bora, Eric Price, and Alexandros G Dimakis. "AmbientGAN: Generttive models from lossy measurements". In: International conference on learning representations. 2018 (cit. on
p. 1).
[8] Zdravko Botev, Joseph F Grotowski, and Dirk P Kroese. "Kernel density estimation via diffusion".. In: (2010) (cit. on 7).
[9] Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. "Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions". In: arXiv preprint arXiv: 1215 (2022) (cit. on p. 2).
[10] Sitan Chen, Giannis Daras, and Alex Dimakis. "Restoration--eeraaatioo Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-type Samplers". In: Proceedings of the 40th International Conference Machine Learning. Ed. by Andreas Krause, Emma Brun- skill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett. Vol. 202. Proceedings of Machine Learning Research. PMLR, 23-29 29 Jul 2023, pp. 4462 4484. URL: mlr wwt.emmrrss..r....lmeeeeeeemmmee wwtp:e/wwssseeeeeeeeeeeeee htt////222////ee html (cit. on p. 2).
[11) Tianyu Chen, Yasi Zhang, Zhendong Wang, Ying Nian Wu, Oscar Leong, and Mingyuan Zhou. Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High- Quality Generation. 2025. arXiv: 2503. 07578 [cs LG]. URL: https: // /ttp///avvvv///a////a//eee//aaaa org/ abs/ 2503.
07578 (cit. on p. 1).
[12] Yunjey Choi, Youngjung Uh, Jacjun Yoo, and Jung- Woo Ha. "StarGAN v2: Diverse image synthesis for multiple domains". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, pp. 8188-8197 (cit. on p. 28).
[13] Xiaoliang Dai et al. Emu Enhancing Image Generation Models Using Photogenic Needles in a Haystack. 2023. arXiv: 2309. 15807 [cs.CV] (cit. on pp. 1, 10).

---

[14] Giannis Daras, Yeshwanth Cherapanamjeri, and Constantinos Costis Daskalakis. "How Much is a Noisy Image Worth? Data Scaling Laws for Ambient Diffusion." " In: The Thirteenth International Conference on Learning Representations. 2025. URL: https: ptttp:://openreveew http/forummidgg2tttPaa..
(cit. on pp. 24).
[15] Giannis Daras, Yuval Dagan, Alexandros G Dimakis, and Constantinos Daskalakis. "Consistent diffusion models: Mitigating sampling drift by learning to be consistent". In: arXiv preprint arXiv:2302.0900 09057 (2023) (cit. on pp. 1, 3)..
[16] Giannis Daras, Mauricio Delbracio, Hossein Talebi, Alex Dimakis, and Peyman Milanfar. "Soft Diffusion: Score Matching with General Corruptions". In: Transactions on Machine Learning Research (2023). ISSN: 3835-8856. URL: https: //tpprreee/eeeeeeeee/// net/forummic w98rebBx1Q (cit. on p. 2).
[17] Giannis Daras, Alexandros G Dimakis, and Constantinos Daskalakis. "Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data". In: arXiv preprint arXiv:2404. 70177 (2024) (cit. on pp. 1, 3).
[18] Giannis Daras, Kulin Shah, Yuval Dagan, Aravind Gollakota, Alex Dimakis, and Adam Klivans. "Ambient Diffusion: Learning Clean Distributions from Corrupted Data". In: Thirty- Conference on Neural Information Processing Systems. 2023. URL: https://opentt evie.. net/forumiid-eee BBB.1id---BBBBKKB (cit. on pp. 1.3).
[19] Mauricio Delbracio and Peyman Milanfar. "Inversion by direct iteration: An alternative to denoising diffusion for image restoration". In: arXiv preprint arXiv:2303.11433 (2023) (cit. on
p. 2).
[20] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. "ImageNet: A large- scale hierarchical image database". In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009, pp. 248-255, DOI: 10. 1109/CVPR. 2009. 5206848 (cit. on p. 28).
[21] Prafulla Dhariwal and Alexander Nichol. "Diffusion models beat gans on image synthesis". In: Advances in neural information processing systems 34 (2021). 1), pp. 8780-8794 (cit. on p. 4).
[22] Sander Dieleman. Diffiusion is spectral autoregression. 2024. URL:
ai/
2024/09 0t////9902///// 2t0////peraaaaaee aaaptrrra--auutoggeessssiie hhmml (cit. on p. 2).
[23] Logan Engstrom, Andrew Ilyas, Benjamin Chen, Axel Feldmann, William Moses, and Aleksander Madry. "Optimizing ml training with metagradient descent". In: arXiv preprint arXiv:2503. /375/ (2025) (cit. on p. 3).
[24] Damien Ferbach, Quentin Bertrand, Avishek Joey Bose, and Gauthier Gidel. "Self-consuming generative models with curated data provably optimize human preferences". In: arXiv preprint 0rXiv:2407.09499 (2024) (cit. on p. 3)
[25] Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. "DataComp: In search of the next generation of multimodal datasets". In: arXiv preprint arXiv:2304. 14108 (2023) (cit. on pp. 1, 3).
[26] Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. GenEval: An Object- Focused Frame- work for Evaluating Text-to-lmage Aligument. 2023, arXiv: 2310 11513 [ca. CV]. URL:
wwwaaiiirgg//...eeee..ee.eooolre aas//ab21110 11513 (cit. on p. 10).
[27] Sachin Goyal, Pratyush Maini, Zachary C Lipton, Aditi Raghunathan, and J Zico Kolter. "Scaling Laws for Data Filtering-Date Curation cannot be Compute Agmostic". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 22702-
22711 (cit. on p. 1).
[28] Dan Hendrycks and Thomas Dietterich. "Benchmarking neural network robustness to common corruptions and perturbations" In: arXiv preprint arXiv:1903. 12267 (2019) (cit. on p. 2).
[29] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. "Gans trained by a two time-scale update rule converge to a local nash equilibrium". In: Advances in neural information processing systems 30 (2017) (cit. on pp. 8, 9).
[30] Jonathan Ho, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models". In: Advances in Neural Information Processing Systems 33 (2020), pp. 6840 -851 (cit. on p. 2).
[31] Danial Hosseintabar, Fan Chen, Giannis Daras, Antonio Torralba, and Constantinos Daskalakis. "DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximization". In: arXiv preprint arXiv:2500. /269 (2025) (cit. on p. 1).
12

---

[32] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. "Adap- tive Mixtures of Local Experts". In: Neural Computation 3.1 (Mar. 1991). _eprint:
1991...1....p...... pp. 79-87. ISSN: 0899-7667. DOT: 10. /162/neco. 1991 1. 1.79. URI.: https://doo..... org.orgg///oo 1100.1166/eec..
1991.3.1. 1.9.9 (cit. on p. 29).
[33] Yiding Jiang, Allan Zhou, Zhili Feng, Sadhika Malladi, and J Zico Kolter. "Adaptive data opti- mization: Dynamic sample selection with scaling laws". In: arXiv preprint arXiv:2410.1 18200 (2024) (cit. on p. 1).
[34] Mason Kamb and Surya Ganguli. "An analytic theory of creativity convolutional diffusion models". In: arXiv preprint arXiv:2412.20222 (2024) (cit. on p. 7).
[35] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. "Elucidating the design space of diffusion- generative models". In: arXiv preprint arXiv:2206. 00364 (2022) (cit. on pp. 8, 29).
[36] Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. "Analyzing and Improving the Training Dynamics of Diffusion Models". In: Proc. CVPR. 2024 (cit. on pp. 2, 9. 24-26, 29).
[37] Tero Karras, Samuli Laine, and Timo Aila. "A style-based generator architecture for generative adversarial networks". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, pp. 4401-4410 (cit. on p. 28).
[38] Sergey Kastryulin, Dzhamil Zakirov, and Denis Prokopenko. PyTorch Image Quality: Metrics and Measure for Image Quality Assessment. Open-source software available at http:/hhwww...oooooooootttesssss photosynthesis tew.teaaaaiaaeee (cit. on p. 23).
2019. URL: https : / github. com /
[39] Sergey Kastryulin, Jamil Zakirov, Denis Prokopenko, and Dmitry V. Dylov. PyTorch Image Quality: Metrics for Image Quality Assessment. 2022. DOT: 10. 48550/ARXXT 2208. 14818.
URL:: https: //axivv////vv../// www./rrgggg.o abs 2208. 14818 (cit. on p. 23).
[40] Varun A Kelkar, Rucha Deshpande, Arindam Banerjee, and Mark A Anastasio. "Ambient- Flow: Invertible generative models from incomplete, noisy measurements". In: arXiv preprint
arXiv:2309.0488 (2023) (cit. on p- 1).
[41] Diederik P Kingma and Jimmy Ba. "Adam: A Method for Stochastic Optimization". In: arXiv preprint arXiv: /41 6980 (2014) (cit. on p. 29).
[42] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment Anything. 2023. arXiv: 2304. 02643 [cV. URL: https: //arxiv.
org/ abs/2304. 02643 (cit. on pp. 10, 29).
[43] Alex Krizhevsky and Geoffrey Hinton. "Learning multiple layers of features from tiny images". In: (2009) (cit. on p. 28).
[44] Jeffrey Li et al. DataComp-LM: In search of the next generation of training sets for language models. 2024, arXiv: 2406. 11794 [ca. (cit. on pp. 1,3).
[45] Tsung- Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zimick, and Piotr Dollár. Microsoft COCO: Common Objects in Context. 2015. arXiv: 1405. 0312 [cs.cv], URL: https: arxiv. org/ abs/ 0312 (cit. on p. 10).
[46] Yvette Lin, Angela F Gao, and Katherine L Bouman. "IMAGING AN EVOLVING BLACK HOLE BY LEVERRAGING SHARED STRUCTURE". In: ICASSP (2024) (cit. on p. 1).
[47] Zeyuan Liu, Zhihe Yang, Jiawei Xu, Rui Yang, Jiafei Lyu, Baoxiang Wang. Yunjian Xu, and Xiu Li. "ADG: Ambient iiffusion-Guider Dataset Recovery for Corruption- Robust Offline Reinforcement Learning". In: arXiv preprint 2r5ivv2505...
(2025) (cit. on p. 1).
[48] Haoye Lu, Qifan Wu, and Yaoliang Yu. "SFBD: Method for Training Diffusion Models with Noisy Data", In: Frontiers in Probabilistic Inference: Learning meets Sampling. 2025. URL.: https:/ / http://ppeeeeveeeee
net/ wwwmm 6d=6HN1 4zuHRb (cit. on p. 1).
[49] Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. "Making a "completely blind" image quality analyzer". In: IEEE Signal processing letters 20.3 (2012), pp. 209-212 (cit. on p. 9). [50] William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. 2023, arXiv: 09748 (cit. on p. 29).
2212. 09748
[cs. CV]. URL: https: // arxii. org/oogg//ao..ooooooooooo
13

---

[51] Guilherme Penedo, Hynek Kydlíček, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf, al. "The Fine Datasets: Decanting the Web for the Finest Text Data at Scale". In: arXiv preprint arXiv:2406. 10066./755 17557 (2024) (cit. on p. 1).
[52] François Rozet, Gérôme Andry, François Lanusse, and Gilles Louppe. "Learning Diffussion Priors from Observations by Expectation Maximization". In: arXiv preprint arXiv:2405.13772 (2024) (cit. on pp. 3).
[53] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. "Laion- 5b: An open large-scale dataset for training next generation image-text models". In: Advances in Neural Information Processing Systems 35 (2022), pp. 25278-25299 (cit. on p. 1).
[54] Vikash Sehwag, Xianghao Kong, Jingtao Li, Michael Spranger, and Lingjuan Lyu. "Stretch- ing Each Dollar: Diffusion Training from Scratch on a dicro--Budget.. In: arXiv preprini arXiv:2407. 158/ / (2024) (cit. on pp. 2, 3, 10, 29).
[55] Kulin Shah, Alkis Kalavasis, Adam R. Klivans, and Giannis Daras. Does Generation Require Memorization? Creative Diffusiio Models using Ambient Diffusion. 2025, arXiv: 2502. 21278
[cs. LG] (cit. on p. 1).
[56] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. "Conceptual Captions: A Cleaned, Hypernymed. Image All-text Dataset For Automatic Image Captioning". In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). Ed. by Iryna Gurevych and Yusuke Miyao. Melbourne, Australia: Association for Computational Linguistics, July 2018, pp. 2556-2565 DOI: 10. N0..8853/vv P18-1238. URL: https: //////caaathhoogyyyy .woggy.orgg wwt/ org/P18-1238/ (cit. on pp. 10, 29).
[57] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously Large Neural Networks: The Sparsely- Gated of
Layer. 2017. arXiv: 1701. 06538 [cs LG]. URL: https: /ttp//aaxvvvv// org/ abs/ 1701. 06538
(cit. p. 29).
[58] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. "Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models". In: arXiv preprint arXiv:22 12. 22.3360 (2022) (cit. on p. 1).
[59] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Gold- stein. "Understanding and Mitigating Copying in Diffusion Models". In: arXiv preprint arXiv:2305..m 20086 (2023) (cit. on p. 1).
[60] Jiaming Song, Chenlin Meng, and Stefano Ermon. "Denoising diffusion implicit models". In: arXiv preprint arXiv:2010.0000 025002 (2020) (cit. on p. 29).
[61] Yang Song and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution", In: Advances in Neural Information Processing Systems 32 (2019) (cit. on p. 2). [62] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. "Scorre-bssed generative modeling through stochastic differential equations". In: arXiv preprint arXiv:201 1.456 (2020) (cit. on p. 2).
[63] Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang. Jifeng Dai, Yu Qiao, Limin Wang, and Hongsheng Li. JourneyDB: A Benchmark for Generative Image Understanding. 2023. arXiv: 2307 00716 [cs.CV] URL: https: / arxiv. wwgg/bs.../....... 223/ .00716 (cit. on pp. 10, 29).
[64] Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Josh Tenenbaum, Frédo Durand, Bill Freeman, and Vincent Sitzmann. "Diffusion with forward models: Solving stochastic inverse problems without direct supervision" In: Advances in Neural Information Processing Systems 36 (2023), pp. 12349-12362 (cit. on p. 2).
[65] Antonio Torralba, Phillip Isola, and William T Freeman. Foundations of computer vision. MIT
Press, 2024 (cit. on p. 2).
[66] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. "Exploring CLIP for Assessing the Look and Feel of Images". In: AAAI. 2023 (cit. on pp. 27).
[67] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. "Exploring clip for assessing the look and feel of images". In: Proceedings of the AAAI conference on artificial intelligence. Vol. 37.
2. 2023, pp. 2555-2550 2563 (cit. on 9).
[68] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. "Image quality assess- ment: from error visibility to structural similarity". In: IEEE transactions on image processing 13.4 (2004), pp. 600-612 (cit. on p. 9).
14

---

[69] Zhou Wang, Eero P Simoncelli, and Alan C Bovik. "Multiscale structural similarity for image quality assessment". In: The Thrity-eventh Asilomar Conference on Signals, Systems & Computers, 2003. Vol. 2. leee. 2003, pp. 1398-1402 (cit. on p. 9).
[70] Zijie J Wang. Evan Montoya, David Munechika, Haoyang Yang. Benjamin Hoover, and Duen Horng Chau. "DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models". In: arXiv preprint arXiv:2210.1 14896 (2022) (cit. pp. 10, 29).
[71] Yasi Zhang, Tianyu Chen, Zhendong Wang. Ying Nian Wu, Mingyuan Zhou, and Oscar Leong. "Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step Sigh-Quality Generation". In: arXiv preprint arXiv:2505. 13377 (2025) (cit. on p. 2).
[72] Yasi Zhang, Tianyu Chen, Zhendong Wang, Ying Nian Wu, Mingyuan Zhou, and Oscar Leong. Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step Fiigh-Qulitty Generation. 2025, arXiv: 2505, 13377 [cs. LG]. URL: https: // /aaxxii.oo. abs/2505, 13377 (cit. on 2).
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our method does any information about the type of corruption, and our experiments show it generalizes to low quality data found in the wild, not just a few artifically controlled corruptions.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We openly discuss the limitations of our approach, such as:
(a) The high and low quality distributions never perfectly merge, so our method always introduces (small) distribution error compared to filtering.
(b) Our method does not work well with certain corruption types, such as masking. These "ill-suited" corruptions require a very large amount noise to merge, such that they are effectively never used during training and our method reduces to filtering in these cases.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Our theorems include premises and assumptions used to prove the result. Informal proofs are found in the main text, referencing formal proofs in the appendix.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All the information the algorithm and the training recipe needed to produce our experiments is included the paper (either in the main text or the appendix). Additionally, make the training and evaluation code public.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All data used is publicly accessible. We release the full training and evaluation code.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the core elements in the main text and the full details in the appendix.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
16

---

Answer: [No]
Justification:
Obtaining error bars would require extremely computationally expensive
retraining of diffusion models.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Computational requirements are provided in the Appendix.
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttp:://neeu.....eee. cc Spppblic/Ethcssuideeeeeee
Answer: [Yes]
Justification: The work does human trials, and all data used is publically avail- able. We analyse the potential negative impacts improving generative model abilities in section B.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See section B.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We not releasing any datasets. We releasing the models. That said, there has already been a model trained and open-sourced from the same dataset. Moreover, our work is not close state-of-the-art text-to-image generation, and thus does not introduce extra risks that do not already exist.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: Prior work has already trained and made public models trained the same data we use to train. Moreover, all datasets are publically available and were introduced by prior research work. which explicitly state and cite.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We do not release any new datasets.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No research with human subjects.
17

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No research with human subjects.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: No important, original, or non-standard usage of LLMs in the paper.
18

---

A Limitations and Future Work
Our work opens several avenues for improvement. On the theoretical side, aim establish matching lower bounds to demonstrate that learning from the mixture distribution becomes provably optimal beyond certain noise threshold. Algorithmically, while method performs well under high-frreuenny corruptions, it remains an open question whether more effective training strategies could be used for different types of corrpptinns (e.g., masking). Moreover, real-world datasets often exhibit patch-wise heterogeneity- example, facial regions are frequently blurred for privacy, leading to uneven corruption across image crops. We plan investigate patch-level noise annotations to better capture this structure in future work. Computationally, the full-version of our algorithm requires the training of classifiers for annotations that increases the runtime. This overhead can be avoided by using hand-picked annotation times based quality proxies as done in our synthetic data experiment. Finally, we believe the true potential of Ambient-o lies in scientific applications, where data often arises from heterogeneous measurement processes.
B Societal Impact
Given that (1) all the datasets used are in the public domain and (2) prior works have already made public models trained on this data, we do not believe our work introduces extra risks that do already exist.
C Theoretical Results
C.1 Kernel Estimation
Assumption C.1. The density is À lipschitz.
Let | set of n independent samples from a density p that satisfies Assumption C.I. Let ý be the empirical density those samples.
We are interested in bounding the total variation distance between Pe p (V(,00, and p. I @ NO, o²). In particular,
n
(
x σ
(x)
(C.1)
no
i=1
where p(u) - 22/2 the Gaussian kernel. We want to argue that the TV distance between V
Po and for is small given sufficiently many samples n. For simplicity, let's fix the support of to be [0, 1]. We have:
2 f (x( - Po
1
L-1
(111/LL
drv
- (x) (
(C.2)
1=0 1/T.
Now let us look at one of the terms of the summation.
f (po(x) -
(41)/L
(111)/z
-o (
+ Po((LL)
Po (x)
(C.3)
/ (po(x) (l(L
(11))
(+1)/L
Ipo(1LL) -p)
(C.4)
11/L
JILL
We first work on the first term. Using Lemma C.6:
(111/L
(1+1)/L
(p=(x) - Po (l(LL)) <
1x - 1/L
(C.5)
1/L
11/L
(C.6)
2
19

---

Next, we work on the second term.
JI1 [1+1)/2
(1
(Pol1L)) - (r(1/L) -
(x))dk (C.7)
( (ir(1/L) (o(
Jyz
JuL 1p-(1/L) - Po (1/D))
(C.8)
According to Lemma C.5, we have that Po is 2 02/2m0
Lipschitz. Then, the second term becomes:
(11))E
- - (x))drr S // (1+1)/1
1 x//dr = 212
(C.9)
1/L
It remains to bound the following term
Jy
(1-1)/2
PPo - (-LL)
JUL
-
(C.10)
We will be applying Hoeffding's Inequality, stated below: Theorem C.2 (Hoeffding's Inequality). Let Y1, ... Yn be independent random variables in (a, b] with mean Then,
n Y,
Pr
p 21 S 2exp (-2nf / ((b (b-
(C.11)
n
Recall that P. be written
xx)/o)
(x()) =
Y
(C.12)
in terms of the random variables Y; = ((x -x)/o)
These random variables are supported in
So, for any x, have that:
V2na
Pr (1) Elp (z)1l 2 < 2exp (-Axo²nt*))
(C.13)
Taking t = V 1og(2L/6) and using the above inequality and the union bound, we have that, with Amarn
probability least 1 8, for all I e {0, I...,, L 1}:
log(2L//5)
(L(LL) - (1/D)( -
(C.14)
4
Let us now compute the expected value of (z).
1
E
[x)]
E
④
(C.15)
n10
σ
i=1
X
(C.16)
no
σ
1 p(u)
x
du = (p NO N(O0, Po(x).
(C.17)
Combining equation C. and equation C.17, get:
20

---

1og(2L/6)
10o(11)) S
(C.18)
4
Putting everything together we have:
X
1
1og(2L/0)
drv -
+
+
2L
2L.²² 2
4TTPn
Choosing L == max 1r we get that:
1
log n + log(1 V A) + log 2/0
drv(Po,Po) Po) x
o2n
C.2 Evolution of parameters under noise
Proof of theorem 4.2: We will use the following facts: Fact 1 (Direct corollary of the optimal coupling theorem). There exists coupling 7 of P and Q which samples a pair of random variables (X, Y) ~ y such that Pr, [X * = drv (P.Q).
Fact 2. For any x, Rd: dTv(W 1),, Ny,, I2)) || 221/20
Proof. The KL divergence between WM4, Σ₁) and N N(2, Ez) is
1 (r(E)"E)) (12 Nr /1) (12 - p1)
Σ2
KI.(((H, (12, E2))
d log
Applying this general result to our case:
|| ||
OL =
0²
We conclude by applying Pinsker's inequality.
corollary of Fact and the optimal coupling theorem the following:
Fact 3. Fix arbitrary æ, E R There exists coupling Yx.y of (0 o²1) and M/ o21), which samples a pair of random variables (Z, Z') Nayy such that Pr [a + Z # y Z'I = || yl1 /20.
Now let us denote by P= P @ ( T) and Q Q s NO, o²)). To establish our claim in the theorem statement, it suffices to exhibit coupling of P and Q which samples pair of random variables (X,X, Y) 7 such that: Pr, XX + YJ < drv Q)
D
We define coupling 7 as follows:
2
1. Sample (X,Y) y (as specified in Fact 1); then
2. sample 7") "x,y (as specified in Fact 3); then
3. output (X, = (X + Y, + +Z)
Let us argue the following:
Lemma C.3. The afooredescribee sampling procedure 7 is valid coupling of P and Q.
Proof. We need to establish that the marginals of P and Q. We will only show that for (X,Y) according the afore-described sampling procedure, the marginal distribution of X is P. as the proof for Y is identical. Since 7 is coupling of and Q, for (X, ~ 7, the marginal distribution of X is P. By Fact 3, conditioning on any value of X and Y, the marginal distribution of is (0, TTT). Thus, X = X + Z, where X P and independently ~ N/0, I2), and thus the distribution of X is P.
Lemma C.4. Under the sefore-dscrite coupling 5: PrşLX + Y dTv(P,Q)) D. 2
21

---

Proof. Notice that, when X = Y, by Fact 3, 2 = Z' with probability 1, and therefore X Y. So for event X # Y to happen, it must be that X * Y happens and, conditioning on this event, that X + Y Z' happens. By Fact 1, Pr, X + YJ drv( Q). By Fact 3, any realization of (, Pr [X + Z * Y + Z IX-XIIl < D where we used that P and Q are supported on a set with diameter D. Putting the above together, the claim follows.
20
C.3 Auxiliary Lemmas
Lemma C.5 (Lipschitzness of the empirical density). For a collection of points X ,....y X(m) consider the function Po (x) Ei=1 xx() where p(u) 42
X
uu? is the Gaussian
kernel. Then Po is
(22/2m)) ( Lipschitz.
Proof. Let us compute the derivative of po:
1 - x(0)
(x) =
(C.19)
dæ
1=1
X(1)
- x2/ (-(x(0-2)/(30) -202((202)
x
exp
(C.20)
V2mna
x²
mnaxexp(-u2/20
(C.21)
V2T02
(C.22)
02V2TT
Lemma C.6 (Lipschitzness of a density convolved with a Gaussian). Let be a density that is X-Lipschitz, Let pa p @ (O, 1). Then, Pa is also X-Lipschitz.
Proof. Let us denote with de (-) the Gaussian density with variance σ². We have that:
(x) pa pv() fivls - 7) ply ())
(C.23)
(p)(z) Po()l fiole -T) - -ply (T)
(C,24)
S A/. y1. foo(r)ar
(C.25)
A/z y/.
(C.26)
D Additional Results
D.1 CIFAR-10 controlled corruptions
Figures 9a, 10, 9b show gaussian blur, motion blur, and JPEG corrupted CIFAR- images respectively different levels of severity. Appendix Table shows results for JPEG compressed data different levels of compression. We also tested our method for motion blurred data with high severity, visualized in the last row Appendix Figure 10), obtaining best FID of 5.85 (compared 8.79 of training only the clean data).
22

---

(a) CIFAR-I0 images corrupted blur at in- creasing levels (Øp = 0.4, 0.6, 1.0).
(b) CIFAR- images corrupted with JPEG com-
pression rates: 25%, 18%, 15% respectively,
Figure 10: CIFAR-10 images corrupted with motion blur at increasing levels of corruption.
D.2 FFHQ-64x64 controlled corruptions
In Appendix 4 we show additional results for learning from blurred data on the FFHQ dataset. Similarly the main paper, observe that our Ambient-o algorithm leads improvements over just using the high-quality data that are inversely proportional to the corruption level.
D.3 ImageNet results
In the main paper, used FID as way measure the quality of generated images. However, FID is computed with respect to the test dataset that might also have samples of poor quality. Further, during FID computation, quality and diversity are entangled. To disentangle the two, we generate images using the EDM-2 baseline and our Ambient-o model and we use CLIP to evaluate the quality of the generated image (through the CLIP-IQA metric implemented in the PIQ package [38, 39]). We present results and win-rates in Table 5. As shown, Ambient-o achieves better per-image quality compared to the baseline despite using exactly the same model, hyperparameters, and optimization algorithm. The difference comes solely from better use of the available data.
23

---

Table 3: Results for learning from JPEG compressed data on CIFAR-10.
Method
Dataset
Clean (%)
Corrupted (%)
JPEG Compression (Q)
mminn Ln
FID
Only Clean
Cifar-10
8.79
15%
1.60
6.67
18%
1.40
6,43
25%
1.27
6.34
Ambient Omni
Cifar-10 10
10
90
50%
1.03
5.94
75%
0.81
5.57
90%
0.63
4.72
Table 4: Results for learning from blurred data, FFHQ.
Method
Dataset
Clean (%)
Corrupted (%)
Parameters Values (o
miin
FID
Only Clean
FFHQ
5.12
10
90
0.8
2.89
4.95
Ambient Omni
FFHQ
10
90
0.6
2.12
4.65
0.4
0.63
3.32
Beyond the scores provided in the main paper, report additional metrics for our Ambient-o XXL+crops model trained on ImageNet in Table These metrics show more evidence for better distribution learning and lower memorization.
D.4 Sanity checks on additive Gaussian noise corruption
As sanity check for the performance of the classifier and the method, we train a classifier data that has actually been corrupted with additive Gaussian noise and we report the average predicted noise level in Table 7. As shown, the model has small errors but is roughly capable of predicting the noise level used. We trained models using estimated noise levels we reproduced the 2.05 FID result reported in the paper "How much is noisy image worth" [14]. Our approach generalizes this idea to arbitrary corruptions, without needing to know the corruption type.
D.5 Effect of number of clean datapoints
For all synthetic corruptions in the paper used 10% clean data. We ablate the effect of this in Table 8. In particular, we provide FID results for training with 1=1%, 5%, 10%, 30% and 50% clean data and (100 x)) blurry data at blur level ag 0.8 on CIFAR-10. As expected, increasing the amount of clean data has very significant impact on the FID. The interesting challenge is how much can we improve the results were the amount of clean datapoints is small.
D.6 Patch level FIDs
The usage of out-of out-of-distribution images for small diffusion times has the risk of introducing small artifacts. We do our best to understand that's the case for the example of using cats images train generative model for dogs, in the paper. To check for artifacts, we report FIDs on the distribution of patches of various sizes for the model trained only on dogs and the model trained on dogs and cats. Results are reported in Table 9, As shown, method also achieves better FID when looking at patches. This is evidence that we have better fine- grained details compared to the model trained only on clean data.
E Ambient diffusion implementation details and loss ablations
Similar to the EDM-2 [36] paper, pre-condition weight balance the importance of different diffusion times. Specifically, we modulate the EDM2 weight X(o) by a factor:
Xamb(a, of ) = 0² (02 - o²nnn²
(E.1)
24

---

Table 5: Additional comparison between EDM-2 XXL and our Ambient-o model using the CLIP IQA metric for image quality assesment. A leads to improved scores despite using the exact same architecture, data and hyperparameters. For this experiment, we use the models with guidance optimized for DINO FD since they are the ones producing the higher quality images.
Metric
EDM-2 [36] XXL
Ambient-o XXL crops
Average CLIP IQA score
0.69
0.71
Median CLIP IQA score
0.79
0.80
Win-rate
47.98%
52.02%
Image (CLIP-IQA=O. 992)
Crop (CLIP-IOAL (QA50.985)
Image (CLIP-IQA=0 069)
Crop (CLIP-IQA=0000 0.01)
Hazer Babst
Fazer Baba
Haver
Baba
Hazer Baba
Hazer Baba Hazer Baba
Image (CLIP-IQA=0. .9.55
Crop (CLIP 10 (..84)
Image (CLP--QQ=00.409)
Crop CCUP-IQA=00006)
Image (CLIP-IQA-OO. (7.778
Crop (CUP-IQA=0.982
Image (CLIP-IQA=0.17))
Crop (CUPP-OA=00000
Image (CLIP-IDA-0.3 0.925)
Crop (CUP-IQA-0.980)
Image (CL/P-QAA00690)
Crop (CUIP-QA=00000
(a) High quality crops
(b) Low quality crops
Figure 11: Results using CLIP to find (a) high-quality and (b) low-quality crops on ImageNet.
25

---

Metric
Ambient-o
Baseline
FID
2.53
2.73
KD
0.04
0.04
CT
27.81
29.07
Precision 93%
92%
Recall
90%
89%
Table 6: Comparison of Ambient-o and EDM-2 baseline performance across metrics. Ambient-o
achieves lower FID and CT, showing better distribution learning and less memorization. It also achieves mild improvements in Precision and Recall.
Gaussian Noise Stddev
Average Annotations
0.05
0.058
0.10
0.
0.20
0.179
Table 7: Predicted noise level when the classifier trained on data that has actually been corrupted with additive Gaussian noise.
Clean Data (%)
FID
1%
21.9
5%
12.9
10%
6.2
30%
2.8
50%
2.4
Table 8: Effect of clean data proportion on FID. Increasing the fraction of clean data substantially improves FID scores.
Patch Size
Baseline FID |
Ours FID
0.3861
0.1825
2
1.1113
0.9761
4
2.9895
1.2498
5.1462
2.7415
16
9,4396
5.8144
32
9.7527
6.5243
64
12.0800
8.9214
Table 9: FID across patch sizes.
for ambient loss based on similar analysis to [36]. We further use buffer zone around the annotation time of each sample to ensure that the loss doesn't have singularities due to divisions by 0. We ablate the precondition term and the buffer size in Appendix Table 10.
For our ablations, we focus on the setting of training with 10% clean data and 90% corrupted data with Gaussian blur of a 0.6. Using ambient pre-conditioning and buffer, obtain FID of 5.56. the same setting, adding the ambient pre-conditioning weight Aumb (a, miin) improves FID by 0.13 points. Next,
ablate two strategies to mitigate the impact of the singularity of
Aamb (a, miin) at a Omin- The first strategy clips the ambient pre-conditioning weight at specified maximum value MAX but still trains for arbitrarily close min. The second strategy also specifies a maximum value, but imposes a buffer
MAX A
min
(E.2)
amb
that restricts training to noise levels such that Aumb(a, a S MAX Clipping the ambient weight to A Anih 2.0 minimally improves FID to 5.35, but clipping to 4.0 significantly worsens it to 5.69. Adding buffer at amb = 2.0 slightly worsens FID to 5.40, but slackening the buffer to 4.0 minimally improves FID to 5.34. We opt for the buffering strategy in favor of the clipping strategy since performance appears convex in the buffer parameter, and because it obtains the best FID.
26

---

Table 10: Ablation study of ambient weight and stability buffer on Cifar- 10 with 10% clean data and 90% corrupted data with blur of 0.6.
Method
FID |
No ambient preconditioning weight and no buffer:
amm (a, m & a Omin
5.49
Adding ambient preconditioning weight:
+ Weight Ammb min) = o² //o" 5 min
5.36
Adding stäbilify buifer//lipping:
+Clip Xumb (0, min) at 2.0
5.35
Clip Aumb (O, min) at 4.0
5.69
Buffer Xamb (a, min)) 2.0 i.. > V220
5.40
Buffer Xamb (a,, min) 4.0 i.. (2/v3)omin
5.34
F
Classifier annotation ablations
We ablate the impact of classifier training data on the setting of (IFAR-10 with 10% clean data and 90% corrupted data with gaussian blur with dB 0.6. When annotating with a classifier trained on the same unbalanced dataset we train the diffusion model on we obtained best FID of 6.04, compared the 5.34 obtained if we train on balanced dataset.
Balanced vs unbalanced data:
Training iterations: We ablate the impact classifier training iterations on the setting of CIFAR-10 with 10% clean data and 90% corrupted data with JPEG compression at compression rate of 18%, training the classifier with a balanced dataset. We report minute variations in the best FID, obtaining 6.50, 6.58, and 6.49 when training the classifier for 5e6, 10c6, and 15c6 images worth of training respectively.
Threshold ablations: An important hyperparameter for the classifier annotations is the threshold used for the approximate mixing of the two distributions. We ablate this Table 11. As shown, the method is relatively robust to miscalibrations. That said, the performance can be improved if this threshold separately tuned for each corruption type and strength.
Threshold
FID
Notes
0.40
6.67
0.43
6.73
0.45
6.43
Baseline (in paper)
0.47
6.21
Best FID
0.48
6.28
Table 11: FID as a function of the classifier threshold. The results obtained here are for CIFAR-10 JPEG corruption at rate 18%. Lower FID values indicate better performance, with the optimal value achieved at threshold 0.47. As shown, the method relatively robust to miscalibrations.
G
Training Details
G.1 Formation of the high-quality and low-quality sets.
In the theoretical problem setting we assumed the existence of good set So from the clean distribution and a bad set Sg from the corrupted distribution. In practice, we do not actually possess these sets initially, but we can construct them so long as we have access Given function on images which tells us wether its good enough to generate or not e.g. CLIP-IQA quality [66] greater than some threshold, we can define our good set SG as the good enough images and Sn as the complement. From this point we can apply the methodology of ambient-o as developed, either employing classifier annotations as our pixel diffusion experiments, or fixed annotations in our large scale ImageNet and text-to-image experiments.
measure of "quality".
27

---

Table 12: Comparison with baselines for training with data corrupted by Gaussian Blur different levels. The dataset used in this experiment is CIFAR-10.
Method
Clean (%)
Corrupted (%)
Parameters Values (rn)
mii
FID
Only Clean
10
8.79
1.0
45.32
No annotations
10
90
0.8
28.26
0
0.4
2.47
2.32
6.95
Single annotation
10
90
0.8
1.89
6.66
0.4
0.00
2.47
10
90
1.0
2.84
6.16
Classifier annotations
10
90
0.8
1.93
6.00
10
90
0.22
2.44
G.2 Details about classifier training details and pitfalls
The sample-dependent annotations lead to stronger experimental performance compared to using a single noise level for all the corrupted samples. However, the developed theory for distribution mixing does not apply here and the of per sample annotations can lead in the distribution. For example, if the target distribution Pn has dogs and cats and the "corrupted" distribution do has only dogs, using per-sample annotations will increase the probability of generating from the dogs class. Hence, we recommend caution when using sample dependent annotations and we leave it for future work how to account for this problem. In what follows, we provide some details about classifier training.
introduction of biases
Crops classifier training details. During training, the classifier takes as input a crop (that can be any size) and tries detect if the crop came from the high-quality distribution the low-quality distribution.
At inference time, we split an image into crops of specific size, let's call it C, and then we see if, on average, they confuse the classifier. The bigger C, the harder is to confuse the classifier. there no confusion for the initial C, decrease it and we try again until we find a crop size for which the classifier is (on average) confused. We underline that, principle, we could do the annotation separately for each crop of the same image, effectively leading to different diffusion times per crop. That said, we used an implementation that averages across crops for simplicity. We finally map from the crop size that confused the classifier to a diffusion time, using Figures 15/17. Higher noise levels require bigger receptive fields (bigger crops) for optimal denoising. So if an image only manages to confuse the classifier small crop, then it is only used for small set of diffusion times.
We also clarify that the diffusion model is never trained on crops of images. We always train using the entire image, even the out-of-distribution images, but for diffusion times small enough (less than mmax) such that for the required receptive field size at that noise level, the target and out-of-distribution data is indistinguishable.
G.3 Datasets
CIFAR-10. CIFAR-10 [43] consists of 60,000 32x32 images of ten classes (airplane, automobile, bird, cat, deer, dog, frog, horse, ship. and truck).
FFHQ. FFHQ [37] consists of 70,000 512x512 images of faces from Flickr. We used the dataset 64x64 resolution for our experiments.
AFHQ. AFHQ [12] consists of 5,653 images of cats, 5,239 images of dogs and 5,000 images of wildlife, for total of 15,892 images.
ImageNet. ImageNet [20] consists of 1,281,167 images of variable resolution from 1000 classes.
28

---

Conceptual Captions.
Conceptual Captions [56] consists of 12M (image url, caption) pairs.
Segment Anything. Segment Anything [42] consists of 11.IM high-resolution images annotated with segmentation masks. Since the original dataset did not have real captions, we use the same LLaVA generated captions created by the MicroDiffusion [54] paper.
JourneyDB. JourneyD consists of 4.4M synthetic image-caption pairs from Midjourney [63].
DiffusionDB consists of 14M synthetic image-caption pairs, mostly generated from Stable Diffusion models [70]. We use the same 10.7M quality-filtered subset created by the MicroD- iffusion paper [54].
DiffusionDB.
G.4 Diffusion model training
CIFAR-10. We use the EDM [35] codebase as reference to train class-conditional diffusion models on CIFAR-IO. The architecture a Diffusion U-Net [60] with -55M paramemeters. We use the Adam optimizer [41] with learning rate 0.001, batch size 512, and no weight decay, While the original EDM paper trained for 200 10t images worth of training, when training with corrupted data we saw best results around 20 x 10t images. On single VxV100 node we achieved a throughput of 0.8s per 1k images, for an average of 4.4h per training run.
FFHQ. Same as for CIFAR-10, 10, except learning was set to 2e 4, we trained for a maximum of 100 10th images worth of training, and best results around 30 10 images worth.
AFHQ. Same as FFHQ
ImageNet. We use the EDM2 [36] codebase a reference to train class-conditional diffusion models on ImageNet. The architecture is a Diffusion U-Net [60] with -125M paramemeters. We use the Adam optimizer [41] with reference learning rate 0.012, batch size 2048, and weight decay, Same as the original codebase, we trained for -2B worth of images. On 32 H200 GPUs, XS models took -3 days to train, while XXL models took -7 days.
MicroDiffusion. We use the MicroDiffusion codebase [54] as a reference to train text-to-image models on an academic budget. We follow their recipe exactly, changing only the standard denoising diffusion loss to the ambient diffusion loss. The architecture a Diffusion Transformer [50] utilizing Mixture-of -frrrrrmmentttts (MoE) feedforward layers [32, 57]. with ~1.1B paramemeters. We use the AdamW optimizer [41] with reference learning rates 2.4c 4/8c - 5/8c 5/8e for each of the four phases and batch size 2048 for all phases. On H200 GPUs, training takes ~2 days to train.
G.5 Classifier training
Classifier training is done using the same optimization recipe (optimizer, learning rate, batch size, etc.) as diffusion model training, except we change the architecture to an encoder-only "Half-Unet", simply by removing the decoder half of the original UNet architecture. The training of the classifier substantially shorter compared to the diffusion training since classification is task is easier than generation.
H Additional Figures
29

---

Figure 12: Uncurated generations from Ambient-o XXL model trained on ImageNet.
30

---

Figure 13: Uncurated generations from our Ambient- XXL model trained on ImageNet.
31

---

Data Availability vs. Noise Level
Amount of data available for training
6000
100% Amount of data available without corruptions
3000
1500
10% Amount available by filtering
2
4
8
10 12
Noise siddev
Figure 14: Amount of samples available at each noise level when training generative model for
dogs in the following setting: (1) we have 10% of the dogs dataset uncorrupted, (2) we have the other 90% the dogs dataset corrupted with gaussian blur with dg = 0.6, and (3) we have 100% of the clean dataset of cats. At low noise levels, the cats, resulting in > 100% of samples available relative to the original dogs dataset size. As the noise level starts increase, gaining some blurry dog samples. As the noise level approaches the maximum all the blurry dogs become available for training, such that the amount of data available approaches 100%.
train on both the high quality dogs and lot of
stop being able to
the out-of-distributien cat samples, but start
Loss vs Context Size
a-005
--85
-525
o=0.55
0.94
0-855
0-0.5
NSE Lost on 2X2 Center Patch
0.03
0-095
0.00
20
25
30
Contest Size
Figure 15: ImageNet-51225 1x25x: denoising loss of an optimally trained model, measured at 2 x center patch, as we increase the context size given to the model (horizontal axis) and the noise level (different curves). As expected, for higher noise, more context is needed for optimal denoising. The large dot each curve marks the point where the loss nearly plateaus.
32

---

Receptive Field Size vs. Noise Level
Erfective Receptive Field Size
0.2
0.6
1.0
Noise Level (o)
Figure 16: ImageNet-512x55 context size needed be within le 3 of the optimal loss for different noise levels. As expected, for higher noise, more context is needed for optimal denoising.
Loss vs Context Size
0-015
--025
05055
0.94
MSE Loss 232 patch
a=085
0-095
0.01
10
Figure 17: FFHQ: denoising loss of an optimally trained model, measured at 2 x 2 center patch, as we increase the context size given the model (horizontal axis) and the noise level (different curves). As expected, for higher noise, more context is needed for optimal denoising. The large dot on each curve marks the point where the loss nearly plateaus.
33

---

Receptive Field Size vs. Noise Level
50
Effective Receptive Field
0.2
0.6
1.0
Noise Level (o)
Figure 18: FFHQ: context size needed to be within le of the optimal loss for different noise levels. As expected, for higher noise, more context is needed for optimal denoising.
Image patches
Probabilty
being inside
Image patches
Probabiiity
inside
611
055
0.00
8:00
6:00
5:0
0.07
0.34
6:1
A76
038
322
C14
0:22
0:0
6:0
091
0.13
341
652
643
(a) Cat image and classification probabilities over patches.
(b) Cat image and classificattin probabiliticss over patches.
Figure 19: Two examples of cats from the AFHQ dataset. We partition each cat into non overlapping patches and we compute the probabilities of the patch belonging dogs classifier trained on patches. The cat on the right has a lot more patches that could belong to dog image according the classifier, possibly due the color or the texture of the fur.
image of dog using cats vs
(a) Cat annotated by a cats vs. dogs classifier that operates with crops of size 8.
(b) Cat annotated by a cats vs. dogs classifier that operates with crops of size 16.
(c) Cat annotated by cats vs. dogs classifier that operates with crops of size 24.
Figure 20: Patch-based annotations of a cat image from AFHQ using cats vs. dogs classifiers trained different patch sizes.
34

---

Image spiit into patcnes
Prodabillty patcn being from class dog.
0.03
0.16
0.00
0.00
0.00
0.00
0.00
0.00
0.31
0.02
0.00
0.00
0.00
0.00
0.00
0.00
0.94
0.26
0.00
0.00
0.00
0.00
0.00
0.00
0.98
0.07
0.00
0.00
0.00
0.00
0.00
0.00
0.02
0.00
0.00
0.00
0.98
0.31
0.00
0.00
0.00
0.01
0.02
0.02
0.73
0.96
0.57
0.02
0.11
0.65
0.94
0.37
0.41
0.72
0.58
0.96
0.28
0.23
0.05
Figure 21: Patch level probabilities for dogness in synthetic image (procedural program). The cat has more useful patches than this non-rcalistic procedural program.
image into 2on7nes
ITCOOCOTY pAEER DRINg mSan clss
inage #12 part
KPSSS
EOETA CORO OOIS
340
009
6:0
0:00
6:2
352
3CB
240
342
302
6:00
0:00
600
325
SES
340
063
0:00
6:0
SEV
6:3
0.0
224
(a) Synthetic image and classification probabilities over patches.
(b) Synthetic image and classification probabilities over patches.
Figure 22: Two examples procedurally generated images. We partition each image into non overlapping patches and we compute the probabilities of the patch belonging to image of dog using a synthetic image vs dogs classifier trained on patches. The image on the right has a lot more patches that could belong dog image according to the classifier, possibly due to the color the texture.
Image SOIE pRIceeS
ITECERIIT
paEce DRITO TTOR
image sput parsoNe
pIRch Seing CIES
038
DEC
EPC
0:2
3.5
GM
DES
60C
4.0
035
(a) Cat image and classification probabilities over patches.
(b) Cat image classification probabilities over patches.
Figure 23: Two examples of cat images. We partition each image into nonoverlapping patches and we compute the probabilities of the patch belonging to an image of wildlife using cats vs wildlife classifier trained on patches. The image on the right has lot more patches that could belong to wildlife image according to the classifier, possibly due to the color or the texture.
35

---

(a) Example batch.
(b) Noisy batch.
Figure 24: Example batch.
(a) Highest quality images from CC12M according
(b) Lowest quality images from CC12M according
CLIP.
to CLIP.
Figure 25: CLIP annotations for quality of images from CC12M.
badmiinggn
Badg!
Bin Laden Cask losk Obama
Grumin
34
(a) Highest quality images from SAIB according to CLIP.
(b) Lowest quality images from SAIB according
to CLIP.
Figure 26: CLIP annotations for quality of images from SA1B.
36

---

Towp
(a) Highest quality images from DifIDB according
(b) Lowest quality images from DifIDB according
to CLIP.
to CLIP.
Figure 27: CLIP annotations for quality of images from DifIDB.
(a) Highest quality images from JDB according to CLIP.
(b) Lowest quality images from JDB according to CLIP
Figure 28: CLIP annotations for quality of images from JDB.
37

---

Image Quality Distribution
0.8
CLIP-0A Quality Score
0.2
Fraction of Dataset (sarted by quality)
Figure 29: Distribution of image qualities according to CLIP for ImageNet-512.
38


Reviews:
[{"review_id": "KySifQfh9C", "reviewer": "Reviewer_gDjV", "summary": "This paper proposes a novel framework for training diffusion model that can leverage low-quality, synthetic and out-of- distribution data, which are often useless and discarded previously, to improve the generation. Focusing on the task of 2D image generation, the paper mainly propose two principles in distinguishing which low-quality data can be used: (1) Low-quality data with very high noise level benefit the training since their distribution is close enough to the high-quality data; (2) Low-quality data with very small nosie level benefit the training after cropping since some local regions within them provide sufficient information as high-quality data. A specific classifier is trained for each technique to determine the threshold of the noisy level above or below which the low-quality data can be used. The paper provides both rigorous theoretical derivation and extensive experiments to show the soundness of (1) and (2). The paper also provides results on the task of training image generative model on ImageNet dataset, demonstrating that their method benefits the pratical application.", "questions": "The author should clarify all the unclearness mentioned in the strengths and weaknesses part.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The problem this paper focuses on is meaningful in broad fields of generative AI, as almost all the datasets contain a large proportion of low-quality data which do not benefit the training. Data filtering and curating is a topic that cannot be gotten around in training large generative models, and it often takes great effort to do it. Meanwhile, there is a trend that scaling up large generative models is more and more difficult because all the high-quality in the world is close to being exhausted. To this point, even though this paper doesn’t show a significant improvement in the practical application with their method, it is inspiring in how to efficiently use data.\n2. The proposed principles are simple and the method is generalizable enough (only rely on training specific classifiers). Though the paper focuses on the 2D image generation task, the principle (1) mentioned in the summary part can be easily generalized to any diffusion model. The principle (2) is a bit stricter but is also supposed to be generalized to the data distribution with the property of locality. \n3. The paper demonstrates the effectiveness of the proposed method well from different perspectives. For example, to prove the principle (1), the paper not only provides a clear and rigorous theoretical derivation to show that mixing low-quality data in high-quality dataset results in a lower upper-bound error, but also shows it with multiple small experiments on various types of corrupted data. Besides, extensive experiments are conducted to demonstrate the points that support the principle (2) (e.g. illuminating the relationship between noise level and context size with multiple plots from different experiments).\n\nWeaknesses:\nI don’t see major weaknesses in this paper. The following are several minor questions:\n1. I didn’t catch how cropping is applied in detail when training a diffusion model. Take sample dependent annotation as an example, from my understanding, the $t_{max}$ for each sample is different but is determined by a same crops classifier. Is the crop size the same and fixed for training such as classifier (i.e. the classifier decides the max noise level of each low-level data sample below which the sample cannot be distinguished with high-level data samples under a specific crop size)? If so, what crop size is used in the experiments and what are the pros and cons of using large or small crop sizes? And more important, how the image is cropped in training the classifier and the diffusion model. Is every crop of one image used and are averaged in terms of the output or are the crops randomly picked?\n2. A small typo: Figure 7 should be Table 7", "was_revised": true, "final_justification": "The rebuttal answered my questions about cropping. Given the overall quality of this paper and the authors' detailed rebuttal, I will maintain my support for this paper to be accepted.", "rebuttal": "We thank the Reviewer for their time and very encouraging feedback! We are very pleased to see that the Reviewer appreciated the novelty of our approach, the breadth of our experimental evaluations, the theoretical proofs and the generality of the method. We most certainly agree with the Reviewer that as we are running out of Internet data, methods for efficient usage of data will become increasingly relevant. The same of course holds for scientific applications where we don't have a lot of data to start with.\n\n**Regarding the question about cropping**: Great question and apologies for not having it clear in the first place in the paper. During training, the classifier takes as input a crop (that can be any size) and tries to detect if the crop came from the high-quality distribution or the low-quality distribution.\n\nAt inference time, we split an image into crops of a specific size, let’s call it C, and then we see if, on average, they confuse the classifier. The bigger the C, the harder it is to confuse the classifier. If there is no confusion for the initial C, we decrease it and we try again until we find a crop size for which the classifier is (on average) confused. We start with C=64 (the whole image), and work our way down. We underline that, in principle, we could do the annotation separately for each crop of the same image, effectively leading to different diffusion times per crop. That said, for the crop experiments we did in the paper, we used an implementation that averages across crops for simplicity.\n\nWe finally map from the crop size that confused the classifier to a diffusion time, using Figures 15/17. Higher noise levels require bigger receptive fields (bigger crops) for optimal denoising. So if an image only manages to confuse the classifier at a small crop, then it is only used for a small set of diffusion times.\n\nWe clarify that the diffusion model is never trained on crops of images. We always train using the entire image, even the out-of-distribution images, but for diffusion times small enough (less than t_max) such that for the required receptive field size at that noise level, the target and out-of-distribution data is indistinguishable.\n\nWe hope this clarifies things, and we will definitely make it clearer in the camera-ready version of our work.\n\n\n**Regarding the typo**: We thank the Reviewer for careful reading! We will fix it in the camera-ready version.\n\n**If there is anything else we can do to further strengthen the Reviewer’s support for the paper, we would be happy to do so.** Once again, we thank the Reviewer for their time and feedback."}, {"review_id": "48jxsiQ82x", "reviewer": "Reviewer_JDMp", "summary": "This paper presents a method to enhance diffusion models using low quality, synthetic, and out of distribution images. It introduces the Ambient Diffusion Omni framework, which leverages spectral power law decay and locality of natural images. Validated on images with Gaussian blur, JPEG compression, and motion blur, it achieves state-of-the-art ImageNet FID and improves text-to-image generation in quality and diversity.", "questions": "1. Specify the weakness: detail how to train a classifier for text-to-image tasks without class label.\n2. Precision and recall metrics are lacking to demonstrate the improvement in diversity.\n3. Why is the FID improvement rather small or even worse? More data causes higher GPU resource consumption, and thus yielding worse results is unacceptable.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. Traditional methods that filter high-quality data before training have a significant advantage in training efficiency. While adding bad data may not always be cost-effective, it can be meaningful if it yields results that cannot be achieved by training solely on high-quality datasets. I believe the most important benefits are the generative diversity and semantic understanding ability gained from retaining more samples. \n\nWeakness\n1. The authors' experiments lack diversity metrics like the commonly used precision and recall in Imagenet.\n2. The concept of \"bad data\" presented in this paper is commendable. However, in practical applications, data are mostly in a curated form, and in real-world scenarios, \"bad data\" characterized by low motion, poor aesthetic metrics, limb distortion, incorrect spatial relationships, misinterpretations, etc., are far more critical. Additionally, the method section predominantly focuses on data corruption rather than these real-world manifestations of bad data. Moreover, with the requirement to train multiple classifiers, the paper fails to clearly demonstrate how the proposed approach can be extended to broader application scenarios.", "was_revised": true, "final_justification": "The rebuttal has addressed most of my concerns, though I remain confused about the classifier.​\nGiven the intriguing motivation, as well as the solid analysis and experiments presented, I will maintain my original score.", "rebuttal": "We thank the Reviewer for their time and feedback!\n\n**Reviewer:** \"Traditional filtering methods have a significant advantage in training efficiency.\"\n\nWe want to point out that it is not necessarily true. For many applications, practitioners run multiple epochs during training. Hence, **the important parameter is the number of training updates, not the number of datapoints**. All the baseline models used in the paper obtain their peak performance after multiple epochs of training. For example, for the ImageNet generative models, the EDM-2 XS baseline sees 2,147,483,000 (2B) images during training from a dataset of size 1.2M images. So in total, there are approximately **1000 epochs** being run. Similarly, our text-to-image baseline (microdiffusion) sees 686,080,000 (700M) images during training, but the dataset only has 37M images (~19 epochs being run). \n\nMethods that leverage low-quality data can run the same number of training updates with fewer repetitions of the same data, leading to less memorization and increased diversity. **For all our experiments, we trained our models for the same number of training steps as the baseline models**. \n\n**Reviewer:** \"experiments lack diversity metrics like the commonly used precision and recall in Imagenet\"\n\nWe thank the Reviewer for their recommendation! For our text-to-image results, we already report Vendi Diversity (Line 320) and we show a **greater than 13% improvement** compared to a baseline trained on a curated dataset. We further show qualitative results for diversity in Figure 29 of the Appendix.\n\nFor our ImageNet results, we initially did not report these metrics because we followed exactly the evaluation protocol from the EDM and the EDM-2 papers that only report FID. **Following the Reviewer’s suggestion, we further report the following metrics**:\n\n* Precision (higher is better): measures the expected likelihood of fake samples against the real data manifold\n* Recall (higher is better): measures the expected likelihood of real samples against the fake data manifold.\n* KD (lower is better): measures the kernel distance between the real and the generated distribution\n* CT (lower is better): A memorization statistic (see A Non-Parametric Test to Detect Data-Copying in Generative Models).\n\nFor all these metrics, we use the dgm-eval library that provides implementations of metrics used in the paper:  Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models\n\nWe report the results for our best model (Ambient-o-XXL+crops) trained on ImageNet (Table 1 in the paper) and the baseline model trained by the EDM2 authors. We use the same training parameters, training updates and inference parameters (EMA=0.015, guidance strength=1.2 – values taken from the EDM2 paper) for both models. The results are shown below:\n\n| Metric        | Ambient-o | Baseline |\n|---------------|-----------|----------|\n| **FID**       | **2.53**  | 2.73     |\n| **KD**        | 0.04      | 0.04     |\n| **CT**        | **27.81** | 29.07    |\n| **Precision** | **93%**   | 92%      |\n| **Recall**    | **90%**   | 89%      |\n\nAs shown, our method improves the FID from 2.73 to 2.53 and it also achieves a reduction in memorization (lower CT score). We further achieve mild improvements in Precision and Recall. \n\nWe further want to clarify that **our ImageNet models did not use extra data** -- we just treat some of the existing data as noisy, and this leads to quality improvements. The improvement in Precision/Recall is mild since these are primarily diversity metrics, and we did not have access to extra data for this experiment.\n\nWe will report all these results in the Camera Ready version, we thank the Reviewer for the recommendation.\n\n**Reviewer:** \"[...] the method section predominantly focuses on data corruption rather than these real-world manifestations of bad data\"\n\nWe believe the Reviewer has a **critical misconception** about our work here. We indeed perform experiments on real-world datasets on which real corruptions (such as poor aesthetics, low lighting, and other bad data forms) appear. In Figure 4 (right) we show examples of such “bad data” on ImageNet. Similarly, in Figures 24, 25, 26, 27 in the Appendix, we show examples of “bad data” that appear on the datasets we used for our text-to-image results: CC12M, SA1B, DiffDB, and JDB, respectively. \n\nOur framework treats these real-world manifestations of bad data as noisy samples, and by doing so, it leads to quality and diversity improvements. For example, in the text-to-image setting, we manage to significantly outperform our micro-diffusion baseline in various benchmarks, such as DrawBench and Partiprompts (Figure 6), as well as in COCO FID and GenEval (Figure 7).\n\nWe only performed synthetic corruptions for the first part of our experimental validation to verify the effectiveness of each component of our approach, prior to moving to real corruption manifestations.\n\n**Reviewer:** \"However, in practical applications, data are mostly in a curated form\"\n\nOur paper questions this curation/filtering process. As we show in our text-to-image results section, filtering/curation leads to a greater than 13% drop in DINO Vendi Diversity. Qualitative results of this drop in diversity are further shown in Appendix Fig. 29. Further, such filtering pipelines are often driven by heuristics and arbitrary quality cutoffs. Instead of throwing away completely some samples, we propose using the “bad” samples only for certain noise times. This paradigm leads to a significant increase in diversity while still producing samples of high-quality (see lines 314-322 in the paper).\n\nFiltering out datapoints entirely makes sense when we are bound by compute, not by data. I.e. if we can only run K training updates and we have at least K high-quality samples, it makes sense to use those. \n\n**Reviewer**: \"[...] the requirement to train multiple classifiers, the paper fails to clearly demonstrate how the proposed approach can be extended to broader application scenarios.\"\n\nWe want to highlight that, in its simplest form, our idea does not require the training of any classifiers. The proposal is the following: instead of filtering out the “bad data”, just use them for very high noise. Similarly, for out-of-distribution data, we propose using them only for “low-noise”. The amount of “high-noise” or “low-noise” can be a single hyperparameter.\nAlready, this very simple idea achieves significant improvements – for the dog-cats experiment, FID improves from 12.08 to 9.85 and for our text-to-image results COCO FID improves from 12.73 (Stable Diffusion v1 level) to 10.61 (DALLE-2 level).\n\nWe provide further experiments to show this by showing FID results without a classifier (fixed sigma, found by hyperparameter tuning) for blurry CIFAR-10 images:\n\n| Setting            | Blur 1  | Blur 0.8 |\n|--------------------|---------|----------|\n| Only clean data    | 8.79    | 8.79     |\n| No annotations     | 45.32   | 28.26    |\n| Classifier         | 6.16    | 6.00     |\n| Fixed sigma        | 6.95    | 6.66     |\n\n As shown, \"Fixed sigma\" approaches the performance of the trained classifier.\n\nThe classifiers provide a more principled way of selecting a different noise level per sample, as they estimate the time at which the TV distance goes below a threshold. The most principled version of our framework requires the training of classifiers over two things: noise levels and crops. The training of the classifiers is computationally much cheaper compared to the training of the diffusion model. However, we acknowledge that this might add engineering complexity, and we encourage practitioners to start with the simplest version of our framework (fixed sigma) using hyperparameter tuning or domain expertise to select the allowed noise levels for each sample.\n\n**Reviewer**: \"detail how to train a classifier for text-to-image tasks without class label.\"\n\nThe training of our classifier only requires the existence of a small set of “top-tier” samples. It is impossible for any method (not just ours) to work without it. The reason is that if the degradation model is unknown and there is no ground-truth sample access, the underlying distribution of pristine samples is not identifiable. \n\n**Reviewer**: \"Why is the FID improvement rather small or even worse? More data causes higher GPU resource consumption [...].\"\n\n**We always train for the same number of training steps as the baseline**, despite the fact that we have more data. Hence, our method is as computationally intensive as the baseline.\n\nRegarding FID: for our text-to-image results, the FID improvement on COCO is massive (from 12.37 to 10.61). That’s the equivalent of going from a Stable Diffusion v1 model to a DALLE-2 model – and this is achieved only by changing the way we use the data (no extra compute + same hyperparameters). \n\nFurther, for ImageNet, we are exposed to the same number of samples as the baseline – we just use some of the available data as low-quality. Despite being exposed to the same data and being trained for the same GPU hours, Ambient-o-XXL+crops achieves state-of-the-art FID across all evaluation metrics and guidance settings. \n\nWe also want to highlight that the test images of ImageNet might also have imperfections – so if our model only produces pristine quality images, it might get penalized in FID.  To account for this, **we further report the mean CLIP IQA score for our Ambient-o-XXL+crops and the baseline XXL model (EDM2) below**:\n\n* Baseline XXL model average CLIP IQA: 0.69\n* Ambient-o-XXL+crops average CLIP IQA: **0.71**\n\nThe increase in the average CLIP IQA corroborates our findings that our model produces samples of higher quality.\n\n**We hope that the additional experiments and the clarifications addressed the remaining concerns. If so, we would highly appreciate it if the Reviewer considers raising their score to signify stronger support for our paper.**"}, {"review_id": "ulCO4QvKdp", "reviewer": "Reviewer_qqMU", "summary": "The paper proposes a novel method, Ambient Diffusion Omni, to learn from bad data. The method can learn clean distributions from bad data without access to the forward operator. Results on Gaussian blur, JPEG, and motion blur verify the effectiveness of the proposed method. Theoretical justification is provided for understanding the tradeoff encountered when learning from corrupted data.", "questions": "1. It would be interesting to see the results of the current method on $y = x + w$, where $A$ is an identity matrix. In this case, the estimated $t_{\\text{min}}$ directly reflects the noise level added to the original image. However, the idea of learning a classifier for noise is not novel, as previous works such as [4] also use GANs at different diffusion steps. The estimated $t_{\\text{min}}$ can also indicate the strength of the proposed classifier.\n\n2. Recent works [1], [3] suggest that using Eq. 2.2 in the paper with the estimated $t_{\\text{min}}$, diffusion models can effectively learn clean distributions by ablating various sampling methods, e.g., truncated sampling. [1] also shows that, even without knowing the true noise level, ablating over noise levels can still yield reasonable performance.\n\n3. The paper lacks ablation studies on the two regimes—high-quality and low-quality. What happens if either regime is removed during training? It would be insightful to see which regime contributes more to performance. Notably, [2] demonstrates that using only Eq. 2.2 and truncated sampling in the noisy setting, or standard diffusion objectives in the noise-free setting, already achieves fairly good results.\n\n4. It would also be interesting to extend the results to $y = A(x) + w$, where $w$ is noise following an unknown distribution, aligning with the setup in the paper.\n\n5. The paper claimed it is learning from bad data, but in the experiments 10% data are necessary, right? It is also interesting to see the performance without clean data. This requirement of clean data would hurt the novelty of this setup. \n\n**Suggestions for clarification:**\n\n1. Provide an ablation on the thresholds used in the paper.  \n2. Discuss the importance of estimating the noise level. What if $t_{\\text{min}}$ were treated as a hyperparameter and optimized via grid search? This might be more effective than training a noise classifier and selecting a threshold, which is labor-intensive.  \n3. Clarify the effectiveness of learning in the low-noise regime. What are the results when training in the high-noise regime with truncated sampling? Are there other techniques beyond cropping that could improve performance?\n\nOverall, the paper presents an interesting setup and approach. However, the lack of ablation studies on key components weakens its empirical foundation. My current rating reflects this limitation.\n\n\n[1] Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation\n\n[2] Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation\n\n[3] How much is a noisy image worth? Data Scaling Laws for Ambient Diffusion\n\n[4] Diffusion-GAN: Training GANs with Diffusion", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper proposes an interesting setup where the forward operator is unknown. \n2. The paper provides theoretical justification for the tradeoff encountered when learning from bad data. \n3. The paper provides experiments on relatively large-scale datasets like ImageNet, which enlarges the impact of this field.\n\nWeaknesses:\n1. The overall storytelling is interesting, but the method proposed is a combination of techniques. \n2. A lack of examination of the power of each component could hurt the originality of the proposed method. \n3. A fair comparison with possible solutions to the proposed interesting setting would weaken the paper.", "was_revised": true, "final_justification": "The authors have addressed my concerns through thoughtful discussions.  I thus increase the rating to 4.", "rebuttal": "We thank the Reviewer for their time and feedback! The Reviewer appreciated our experimental setup, our theoretical proofs and the strength of our experimental results. There are some concerns regarding missing ablations. We ran **numerous ablations to address these concerns**:\n\n* We show results without training classifiers.\n* We provide results on additive Gaussian noise.\n* We show results with truncated sampling.\n* We provide baseline results for only learning with clean data on ImageNet.\n* We experiment with other corruption processes with additive noise.\n* We test the robustness of the thresholds.\n\n**Reviewer**: \"[...] the method proposed is a combination of techniques. What if the noise level were treated as a hyperparameter and optimized via grid search?\"\n\nAt its core, our method is very simple: bad data should only be used in a subset of diffusion times. Blurry images, for example, are typically useful in the high-noise regime, while out-of-distribution images might be useful in the low-noise regime. The training of the classifiers provides a rigorous way of finding the “allowed training times” (as they estimate the TV distance), but this is not necessary. For example, for our controlled experiments on blurry images, annotating all the corrupted samples with the same value already yields pretty strong results. We show these results below (annotated with Fixed Sigma):\n\n| Setting | Blur 1 | Blur 0.8 |\n|-|-|-|\n| Only clean data | 8.79 | 8.79 |\n| No annotations | 45.32 | 28.26 |\n| Classifier | 6.16 | 6.00 |\n| Fixed sigma | 6.95 | 6.66 |\n\nOur paper has similar results (Fixed Sigma) for the out-of-distribution data experiment (Table 2b), too. Further, for our text-to-image results, we similarly annotated all the data from DiffDB at the same noise level.\n\n**Reviewer**: \"It would be interesting to see the results of the current method on the additive Gaussian noise case. It can also indicate the strength of the classifier.\"\n\nWe agree with the Reviewer that this is a good sanity check! We corrupt our data with the noise levels $\\sigma=0.05, 0.1, 0.2$, as in the paper \"How much is a noisy image worth\", and we annotate each datapoint with the classifier.\nThe estimated noise levels are shown below:\n\n| Gaussian Noise Stddev | Average Annotations |\n|-|-|\n| 0.05 | 0.058 |\n| 0.10 | 0.102 |\n| 0.20 | 0.179 |\n\nAs shown, the classifier does a pretty reasonable job of estimating the noise level. We trained models using the estimated noise levels and we reproduced the $\\approx 2.05$ FID result reported in the paper \"How much is a noisy image worth\". Our approach generalizes this idea to arbitrary corruptions.\n\n**Reviewer**: \"Recent works [1], [3] suggest that [...] diffusion models can effectively learn clean distributions by ablating various sampling methods, e.g., truncated sampling.\"\n\nWe believe the Reviewer has a **critical misunderstanding** here. In these works, it is known that the samples have been corrupted by Gaussian noise (and the noise level is given or can be estimated). However, for unknown degradation models (like the ones we deal with in the paper) it is information-theoretically impossible to recover the uncorrupted distribution without access to clean samples from it. That might be better understood with an example. If we are just given blurry samples, it is impossible to know whether the underlying distribution has blurry samples or the underlying distribution is a clean distribution and we just happened to observe blurry observations from it. Recovery guarantees for the underlying distribution without access to clean samples are only possible if the degradation model is known – hence, the distillation method of [1, 2] wouldn’t work for our settings where the corruption model is unknown. That said, we sincerely thank the Reviewer for bringing them to our attention, and will ensure they are properly cited in the camera-ready version.\n\n**Reviewer:**: \"The paper claimed it is learning from bad data, but in the experiments 10% data are necessary, right? It is also interesting to see the performance without clean data. This requirement of clean data would hurt the novelty of this setup.\"\n\nGiven only access to blurry samples and no prior on the degradation model, the best our method can achieve is to recover the distribution of the blurred samples. The FID of the blurred training set is given below:\n\n| $\\sigma_B$ | Blurred Data FID | Only Clean Data FID | Our Method FID |\n|-|---------------|---------------------|----------------|\n| **1.0**    | 53.11         | 45.32                | **6.16**       |\n| **0.8**    | 32.09         | 28.26                | **6.00**       |\n| **0.6**    | 11.84         | 11.42                | **5.34**       |\n\nOur method achieves significantly better FIDs (6.16, 6.00 and 5.34 respectively), with the benefit coming from the usage of the clean data. On the other hand, just using clean data does not suffice (see Table 2a) as it gives FIDs 45.32, 28.26 and 11.42 respectively. Hence, for the success of our method the combination of a small set of clean data and a large set of corrupted data is essential. These results are similar in spirit to the results obtained by the paper How much is a Noisy Image Worth that the reviewer mentions. Our work generalizes these results to arbitrary degradations (beyond additive Gaussian noise).\n\n**Reviewer**: \"The paper lacks ablation studies on the two regimes—high-quality and low-quality. What happens if either regime is removed during training? It would be insightful to see which regime contributes more to performance.\"\n\nSee above for results when only low-quality data is available. As we explained above, the underlying distribution is not identifiable if the degradation model is not known and the best one can hope for is recovering the corrupted distribution – this leads to very poor FID scores.\n\nFor the regime where we are only given high-quality data, we actually have results in the paper and we think that the Reviewer might have missed them. This is the Only Clean baseline in Table 2a and the Additional Data = None rows in Table 2b. We further have results for fine-tuning the text-to-image model only on a curated high-quality subset in lines 318-322 of the paper, showing that our method achieves a comparable FID score with a very significant increase in diversity (>13%).\n\nBased on the Reviewer’s recommendation, we further report results on ImageNet for the Only Clean data baseline. We report results when 10% and 50% of ImageNet is available. We show test FID below:\n\n\n| Method                    | FID   | FD (DINO) |\n|-|-|-|\n| Baseline XS               | 3.77  | 115.16    |\n| Ambient XS                | **3.69**  | **115.02**    |\n| Only Clean Data (10%) | 58.09 | 1001.79   |\n| Only Clean Data  (50%) | 9.77  | 364.44    |\n\nAs shown, having access to low-quality data is critical because if there is not enough high-quality samples, performance is quite poor. We will include these results in the Camera Ready of our work – we thank the Reviewer for suggesting this.\n\n**Reviewer**: \"It would also be interesting to extend the results to y=Ax + w [...]\"\n\nWe thank the Reviewer for their suggestion, and include results on CIFAR-10 for A = gaussian blur and w = zero-mean poisson noise, as well as salt and pepper corruption. As in the paper, we compare results using 10% clean data to using 10% clean data + 90% corrupted data.\n\n\n| Setting                                                                                       | FID   |\n|-|-|\n| Only clean data                                                                                | 8.79  |\n| Gaussian blur $\\sigma_B = 0.6$, $w \\sim \\text{Poisson}(0.2) - 0.2$, classifier annotations     | **5.78**  |\n| Gaussian blur $\\sigma_B = 0.4$, $w \\sim \\text{Poisson}(0.1) - 0.1$, fixed $t_{\\min} = 3$       | **5.72**  |\n| └─ Fixed $t_{\\min} = 1$                                                                        | 5.94  |\n| └─ Fixed $t_{\\min} = 2$                                                                        | 5.72  |\n\n\nFor the salt and pepper, we pre-processed the data with a 3x3 median filter to convert the unbounded impulse noise to a blurring corruption, which is more easily confused by gaussian noise. This allowed us to obtain a best FID of 5.70 using a fixed annotation t_min=3.\n\n**Reviewer**: \"Provide an ablation on the thresholds\"\n\nWe show FID vs threshold results in the JPEG 18% case, keeping the data and classifier constant and changing only the threshold\n\n| Threshold ($\\mathsf{Th}$) | FID   | Notes                  |\n|-|-|-|\n| 0.40                      | 6.67  |                        |\n| 0.43                      | 6.73  |                        |\n| 0.45                      | 6.43  | Baseline (in paper)    |\n| 0.47                      | **6.21**  | Best FID               |\n| 0.48                      | 6.28  |                        |\n\n**Reviewer**: \"What are the results when training in the high-noise regime with truncated sampling? Are there other techniques beyond cropping that could improve performance?\"\n\nTruncated sampling is known to deteriorate results, as the samples get more blurry. We reproduce the same finding. On ImageNet, the results with truncated sampling are shown below:\n\n| $\\sigma$     | FID        |\n|-|-|\n| 0.0          | **2.53**   |\n| 0.1          | 2.86       |\n| 0.2          | 4.50       |\n| 0.4          | 6.87       |\n| 0.8          | 19.89      |\n| 1.0          | 31.41      |\n\nAnother technique that could improve results would be the consistency loss of the CDM paper. However, this loss is expensive to run as it requires sampling trajectories during training.\n\n**We believe our numerous ablations and clarifications strongly address the Reviewer’s concerns. If so, we would be grateful if the Reviewer considers increasing their score (which is currently the lowest among all Reviewers)**"}, {"review_id": "dYZYbzZOHp", "reviewer": "Reviewer_whhj", "summary": "Ambient Diffusion Omni (Ambient-o) proposes a two-stage strategy for training diffusion models on mixed-quality datasets. At large diffusion noise it adds Gaussian noise until a time-aware classifier can no longer tell clean from degraded images, then trains with every sample, at small noise it trains only on local crops whose statistics, chosen by a crop-classifier, match the target distribution. A total-variation contraction proof shows that noise shrinks distribution gaps, giving a formal bias–variance trade-off.", "questions": "1. How sensitive are the final generative results to small variations in the threshold τ used for determining t? Have you tested different τ values (e.g., 0.4 or 0.45) on the same dataset to measure stability?\n\n2. For ImageNet, you used CLIP-IQA to split into high or low quality. In domains where no pre-trained quality estimator exists, what is a practical approach to form S_G or S_B? Could you use a small human-labeled \"gold\" set, or would self-supervised heuristics suffice?\n\n3. You note that degradations affecting low-frequency channels fog, color shifts are \"ill-suited\" since Gaussian noise does not sufficiently confuse them. Have you tried combining Gaussian noise with other synthetic corruptions to push corrupted samples into a \"usable\" regime at lower σ?\n\n4. While Ambient-o improves global image quality and diversity, have you quantified whether it preserves very fine-grained details better or worse than training on fully curated data? For instance, does the patch classifier inadvertently retain tiny out-of-distribution artifacts?", "limitations": "The authors discuss the limitation in the methodology section.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "S1. The paper formalizes how additive Gaussian noise contracts total variation distance between clean and corrupted distributions, quantifying the bias–variance trade-off when including low-quality data.\n\nS2. Unlike prior work on Ambient Diffusion or EM-based approaches that assume access to the exact corruption operator, Ambient-o only requires 1). a small \"clean\" subset and 2). the ability to train classifiers to distinguish clean or corrupted (global or patch-level).\n\nS3. The authors supply ablation studies of Ambient weight and buffer clipping and show how a sample-dependent classifier threshold improves performance over a fixed σ.\n\nS4. Because quality decisions rely only on learned classifiers, the method works with blur, JPEG, motion blur, and even out-of-distribution examples such as using cat images to help a dog model.\n\nW1. Adding noise cannot fully eliminate low-frequency degradations (e.g., color shifts, masking), so the approach may revert to simple filtering in those cases\n\nW2. Performance hinges on the accuracy and thresholding of the quality and crop classifiers; mis-calibration could hurt results, yet sensitivity analyses are limited.\n\nW3. Sample-wise annotations, crop-level decisions, and dual noise schedules introduce extra engineering steps that may slow adoption in production settings.\n\nW4. Ambient-o assumes access to a small set S_G of known high-quality images and a set S_B of known corrupted/out-of-distribution images or a way to label them. In domains lacking any semi-trusted clean subset, it is unclear how to obtain S_G or S_B without manual curation.", "was_revised": true, "final_justification": "Thank you for the reply, which addressed most of my concerns. I have determined my final rating based on this.", "rebuttal": "We would like to thank the Reviewer for their thoughtful feedback. The Reviewer appreciated our formalization of the problem, our theoretical results, the generality of our method, and the breadth of our experiments. The Reviewer has concerns on three fronts:\n\n1) Missing ablation studies on the sensitivity of the classifiers and the size of the ground truth set needed for classification.\n2) Missing evaluations to measure how much we can preserve fine details.\n3) the applicability of our method to low-frequency corruption types.\n\nWe present **numerous new ablation results addressing each one of the raised issues**.\n\n## Sensitivity Results \n\n**Reviewer:** \"How sensitive are the final generative results to small variations in the threshold?\"\n\nGreat question: short answer, results are not very sensitive to miscalibration. To test this, we report FID results for JPEG compression at 18% compression rate (Table 3 of the paper) for a fixed classifier annotating at different thresholds. The results are shown below:\n\n| Threshold | FID | Notes |\n| - | - | - |\n| – | 8.79 | Only clean data |\n| 0.40 | 6.67 | |\n| 0.43 | 6.73 | |\n| 0.45 | 6.43 | Paper result |\n| 0.47 | 6.21 | **Best result** |\n| 0.48 | 6.28 | |\n\n\n**Reviewer**: \"Performance hinges on the accuracy and thresholding of the quality and crop classifiers; mis-calibration could hurt results, yet sensitivity analyses are limited.\"\n\nBeyond the threshold ablations we presented above, we further ablate the size of the dataset needed for classification and the effect of training iterations.\n\n### Training Samples Ablation\n\nWe train two CIFAR-10 classifiers for the Gaussian blur corruption at $\\sigma_B=0.6$. The first classifier is trained using 25K clean samples and 25K blurry samples and the second is trained with 5K clean samples and 45K corrupted samples. The scores are shown below:\n\n| Classifier Training Data | FID |\n| - | - |\n| 25K clean and 25K blurry | 5.34 |\n| 5K clean and 45K blurry | 6.04 |\n\n\nAs shown, the number of samples has an effect on the performance of the classifier, but still, the performance is much better than the 8.79 FID we get by only training on clean samples. Hence, miscalibration due to small training size can hurt a bit the performance, but not to a catastrophic extent.\n\n### Training Iterations Ablation\n\nAnother way to measure sensitivity to miscalibration is to check what happens when we undertrain or overtrain the classifier. The results are shown below for the JPEG at 18% compression rate (Table 3 of the paper):\n\n| Classifier Training Size | FID |\n| - | - |\n| 5M images (100 epochs) | 6.50 |\n| 10M images (200 epochs) | 6.58 |\n| 15M images (300 epochs) | 6.49 |\n\n**We believe the results above prove the robustness of our classifier to various types of miscalibrations.**\n\n### Removing the classifier\n\n**Reviewer:** \"Sample-wise annotations, crop-level decisions, and dual noise schedules introduce extra engineering steps that may slow adoption in production settings\"\n\nWe agree with this point; however, it might not be needed.\n\nAt its core, our method is very simple: bad data should only be used in a subset of diffusion times. Blurry images, for example, are typically useful in the high-noise regime, while out-of-distribution images might be useful in the low-noise regime. The training of the classifiers provides a rigorous way of finding the “allowed training times” (as they estimate the TV distance), but this is not necessary. For example, for our controlled experiments on blurry images, annotating all the corrupted samples with the same value already yields pretty strong results. We show these results below (annotated with Fixed Sigma):\n\n| Setting | Blur 1 | Blur 0.8 |\n| - | - | - |\n| Only clean data | 8.79 | 8.79 |\n| No annotations | 45.32 | 28.26 |\n| Classifier | 6.16 | 6.00 |\n| Fixed sigma | 6.95 | 6.66 |\n\n\nOur paper has similar results (Fixed Sigma) for the out-of-distribution data experiment (Table 2b), too. Further, for our text-to-image results, we similarly annotated all the data from DiffDB at the same noise level without requiring the training of a classifier.\n\n**Reviewer:** \"In domains where no pre-trained quality estimator exists, what is a practical approach to form S_G or S_B? Could you use a small human-labeled \"gold\" set, or would self-supervised heuristics suffice?\"\n\nExcellent question! The problem of sampling from the \"uncorrupted\" distribution is not identifiable if the corruption model is unknown (as in our setting) and we don't have samples from the ground truth. Hence, a \"gold\" set needs to be formed, either with human supervision or with some other domain-specific knowledge. For example, in proteins, there are corrupted data in the Protein Data Bank, but they come with some \"confidence level\" (resolution of the protein). In robotics, there are ground truth data from humans and \"corrupted data\" from simulations. If we are not in such a setting, then human annotation is required to form a \"gold\" set. The good news is that our ablations above suggest that the \"gold set\" can be small without hurting the performance too much (see sensitivity analysis).\n\n## Low-frequency corruptions\n\n**Reviewer:** \"Adding noise cannot fully eliminate low-frequency degradations (e.g., color shifts, masking), so the approach may revert to simple filtering in those cases. Have you tried combining Gaussian noise with other synthetic corruptions to push corrupted samples into a \"usable\" regime at lower sigma?\"\n\nExcellent question! It is true that for certain cases, classifying under Gaussian noise corruption will essentially revert to filtering. Hence, there are two options. Option 1 is to do some preprocessing of the data, as the Reviewer proposes. This can actually take us a long way in certain cases. We experimented with two types of corruptions to show this:\n\n- salt and pepper noise, where we set 5% of the pixels independently to either 0 or 1.\n- random masking, where we delete 70% of the pixels independently.\n\nFor both of these cases, we required very high sigma to mask the corruptions -- essentially reverting to filtering. However, a simple preprocessing of the data dramatically increased performance. For salt and pepper we used a 3x3 median filter and obtained an FID of $5.70$ (compared to the filtering $8.79$) using a fixed annotation of $\\sigma=3$. Similarly, for the random masking, we do a local average of the non-masked pixels (simulating the effect of blurring) and that leads to an FID of $6.72$. These experiments show that with some simple preprocessing we can actually extract more signal out of the available data.\n\nHowever, this is not always possible. For some corruptions, there aren't really easy preprocessing steps we can do to make them amenable to Gaussian noising. If the dataset has a lot of samples with such corruptions, the best way forward in our opinion is to train an \"exotic\" diffusion model (similar to Soft Diffusion / Cold Diffusion and certain flow matching models). These diffusion generalizations learn to connect arbitrary source and terminal measures -- e.g. the fully fogged distribution to the distribution of clean images. If we are willing to train such a model, we can corrupt the samples (not by adding noise, but by e.g. adding fog) to bring them along that exotic-diffusion trajectory. That said, we think this approach will require a lot of engineering efforts and hence we did not experiment with it in the paper.\n\n## Evaluation regarding preservation of very fine features\n\n**Reviewer**: \"have you quantified whether Ambient-o preserves very fine-grained details better or worse than training on fully curated data? For instance, does the patch classifier inadvertently retain tiny out-of-distribution artifacts?\"\n\nSuper interesting question -- we thank the Reviewer for raising this. As in the low-quality data case, using data from any distribution other than the target will have to introduce some artifacts since convergence (even for small crop sizes) is not perfect. However, these artifacts do not seem to be detectable, at least using the tests we performed during the rebuttal. \n\nLet's take the dogs and cats example, which is the most extreme case of using out-of-distribution data. To measure to what extent we preserve fine details, we measure a per-patch size FID: i.e we compare the distribution of patches of the generated samples with the distribution of patches from the training set and we compute the FID at different patch levels. We compare two models: the baseline model trained only on dogs vs our Ambient-o model trained on the same number of dogs + crops from cats. The results for the per-patch FID are shown below:\n\n| Patch Size | Baseline FID | Ours FID |\n| - | - | - |\n| 1 | 0.3861 | 0.1825 |\n| 2 | 1.1113 | 0.9761 |\n| 4 | 2.9895 | 1.2498 |\n| 8 | 5.1462 | 2.7415 |\n| 16 | 9.4396 | 5.8144 |\n| 32 | 9.7527 | 6.5243 |\n| 64 | 12.0800 | 8.9214 |\n\n\nAs shown, our method not only achieves superior FID on the global images, but also achieves better FID when looking at patches. This is evidence that we have better fine-grained details compared to the model trained only on clean data.\n\nTo further assess whether this makes sense, we also computed the per-patch size FID between 1) disjoint training images of dogs and 2) training images of cats and training images of dogs.  Results are shown below:\n\n| Patch Size | FID (Dogs 1 + Dogs 2) | FID (Dogs 1 + Cats 1) |\n| - | - | - |\n| 64 | 29.40 | 176.85 |\n| 32 | 42.94 | 170.83 |\n| 16 | 58.08 | 83.66 |\n| 8  | 26.44 | 37.99 |\n| 4  | 13.88 | 12.37 |\n| 2  | 5.10  | 5.43  |\n| 1  | 0.86  | 1.60  |\n\n\nAs shown, for crop sizes 4 and lower the FIDs are comparable. Our crop classifier assigns an average annotation at crop size 6 (between 8 and 4), which checks out with this experiment.\n\n**We believe the provided experiments and clarifications strongly address the Reviewer's concerns. If so, we would appreciate if the Reviewer considers increasing their rating to reflect this.**"}]

Author rebuttal:

Meta-review: **Meta-Review**

A common practice in training large-scale foundational generative models is to rely on high-quality data, typically by filtering out low-quality or "bad" samples early in the pipeline. In image generation, such samples are often noisy or corrupted in some way.

This paper proposes two techniques for handling noisy data, depending on the degree of corruption:  
(i) leveraging the diffusion process itself, since it already relies on a noise-to-denoising trajectory to benefit from highly noisy images during high-noise steps of diffusion training;  
(ii) using an external classifier to crop and retain high-quality regions from partially noisy images, allowing the model to benefit still from otherwise discarded data.

**Weaknesses:**  
Some concerns were raised regarding the use of a classifier in the cropping-based technique. I agree that this second technique is less novel; leveraging classifiers for data filtering is relatively common, and it still involves discarding parts of the data. Additionally, it depends on an external model. I strongly recommend that the authors include the relevant rebuttal discussions in the final version, clarifying how the classifier is trained and including ablation studies that isolate the performance of the first technique alone, as suggested by the reviewers.

**Recommendation**: Accept (highlight)  
I find the approach, especially the use of the diffusion process itself to handle noisy examples, both interesting and significant. As high-quality data becomes increasingly scarce, methods that employ imperfect data are of high relevance to the community.


**Discussion Summary:**

**gDjV** raised questions about cropping and classifier usage. The authors clarified the procedure, explained that training always uses full images, and added ablations without classifiers. Concerns were resolved, and support remained strong.  **JDMp** questioned diversity metrics, practical “bad data” handling, classifier efficiency, and small FID gains. Authors added precision/recall, CT, KD results, clarified real-world corruptions, and compared fixed-sigma vs. classifiers. Most points were addressed, though skepticism about classifiers persisted.   **qqMU** noted missing ablations and reliance on classifiers vs. fixed sigma, and asked about corrupted-only data. Authors provided extensive ablations (thresholds, clean-data proportion, Gaussian noise, and truncated sampling) and demonstrated that corrupted-only methods cannot recover clean distributions. Rating increased after rebuttal.   **whhj** raised concerns on classifier sensitivity, forming clean subsets, low-frequency corruptions, and fine-detail preservation. The authors presented sensitivity studies, practical routes for clean subsets, preprocessing experiments, and patch-level FID for details. Most issues were resolved, and the score remained positive.
Decision: Accept (spotlight)