Boomerang: Local sampling on image manifolds using diffusion models
Authors: Lorenzo Luzi
Ali Siahkoohi
Paul M Mayer
Josue Casco-Rodriguez
Richard Baraniuk
Venue: Submitted to ICLR 2023 (2023)
Keywords: Diffusion models
local sampling
image manifolds

Abstract: Diffusion models can be viewed as mapping points in a high-dimensional latent space onto a low-dimensional learned manifold, typically an image manifold. The intermediate values between the latent space and image manifold can be interpreted as noisy images which are determined by the noise scheduling scheme employed during pre-training. We exploit this interpretation to introduce Boomerang, a local image manifold sampling approach using the dynamics of diffusion models. We call it Boomerang because we first add noise to an input image, moving it closer to the latent space, then bring it back to the image space through diffusion dynamics. We use this method to generate images which are similar, but nonidentical, to the original input images on the image manifold. We are able to set how close the generated image is to the original based on how much noise we add. Additionally, the generated images have a degree of stochasticity, allowing us to locally sample as many times as we want without repetition. We show three applications for which Boomerang can be used. First, we provide a framework for constructing privacy-preserving datasets having controllable degrees of anonymity. Second, we show how to use Boomerang for data augmentation while staying on the image manifold. Third, we introduce a framework for image super-resolution with 8x upsampling. Boomerang does not require any modification to the training of diffusion models and can be used with pretrained models on a single, inexpensive GPU.

Under review as a conference paper at ICLR 2023
BOOMERANG: LOCAL SAMPLING ON IMAGE MANIFOLDS USING DIFFUSION MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Diffusion models can be viewed mapping points in high-dimensional latent space onto a low-dimensional learned manifold, typically an image manifold. The intermediate values between the latent space and image manifold can be inter- preted as noisy images which are determined by the noise scheduling scheme em- ployed during pre-training. We exploit this interpretation to introduce Boomerang, a local image manifold sampling approach using the dynamics of diffusion mod- els. We call Boomerang because first add noise it closer to the latent space, then bring it back to the image space through dif- fusion dynamics. We use this method to generate images which are similar, but nonidentical, to the original input images on the image manifold. We are able set how close the generated image is the original based on how much noise we add. Additionally, the generated images have degree of stochasticity, allow- ing us to locally sample as many times as we want without repetition. We show three applications for which Boomerang can be used. First, we provide frame- work for constructing privacy-preserving datasets having controllable degrees anonymity. Second, we show how to use Boomerang for data augmentation while staying on the image manifold. Third, we introduce framework for image super- resolution with 8x upsampling. Boomerang does not require any modification to the training of diffusion models and can be used with pretrained models on a single, inexpensive GPU.
input image, moving
Back to = 0 prompt "person"
Back to t = prompt "person"
Back to f = 0 prompt = "person"
Back to to = 0 prompt "person"
Back to = 0 prompt "cat"
Initial Image
Forward 200
Forward 500
Forward to 700
Forward 800
Figure 1: Boomerang via Stable Diffusion (Rombach et al., 2022). Code available here. Starting from an initial image Xp ~ p(Xp), we add varying levels of noise to the latent variables according to the noise schedule of the forward diffusion process. Boomerang maps the noisy latent variables back to the image manifold by running the reverse diffusion process starting from the reverse step associated with the added noise. The resulting images are local samples from the image manifold, where the closeness is determined by the amount of added noise. While Boomerang here is applied the Stable Diffusion model, is applicable other types of diffusion models, e.g., denoising diffusion models (Ho et al., 2020). Additional images are provided in Appendix A.1.

---

Under review as a conference paper at ICLR 2023
1 INTRODUCTION
Generative models have seen tremendous rise in popularity and applications over the past decade, ranging from image synthesis, audio generation, out-of-disrriuution data detection, and reinforce- ment learning to drug synthesis (Kingma & Welling, 2014; Goodfellow al., 2014; Bond-Tayle et al., 2021). One of the key benefits of generative models is that they can generate new samples from an unknown probability distribution from which we have samples. Recently, with the ad- vent of models such as Dall-E 2 (Ramesh et al., 2022), Imagen (Saharia et al., 2022a), and Stable Diffusion (Rombach et al., 2022), a family of generative models known as diffusion models have gained attention in both the academic and public spotlights. A key difference between diffusion models and previous state-of-the art generative models, such as Generative Adversarial Networks (GANs) (Goodfellow et al., 2014), is that diffusion models take fundamentally different approach to image synthesis, opting for a series of denoising objectives, whereas GANs opt for saddle-point objectives that have proven to be difficult to train (Yadav et al., 2018; Mescheder et al., 2018; Dhari- wal & Nichol, 2021).
Generative models estimate the underlying probability distribution, or manifold, of dataset. As such, generative models should be able to produce samples close to a particular data point-known that they have some knowledge about properties of the
as local sampling on the manifold- data manifold. One application of local sampling is to remove noise from an image, especially in the cases of severe noise, where traditional denoising methods might fail (Kawar et al., 2021). Another application of local sampling is data augmentation, which involves applying transformations onto copies of data points to produce new data points. These data points are still from the original data distribution, but different enough from existing data to encourage the model to generalize in downstream tasks such as classification (Wong et al., 2016). While established techniques image augmentation include crops, flips, rotations, and color manipulations, data augmentation techniques for images and other data types are an ongoing field of research (Yang et al., 2022; Wen et al., 2021; Feng et al., 2021).
Diffusion models are typically designed to sample globally on the image manifold instead of per- forming local sampling. GANs (Goodfellow et al., 2014), VAEs (Kingma & Welling, 2014), and NFs (Rezende& & Mohamed, 2015) can sample locally to a limited extent. VAEs and NFs can project data point x into latent vector in their respective latent spaces and then re-project back into the original data space, producing an estimate x' of the original data point. The key difference between XYVE and x is that neither the encoder nor the decoder in VAE is invertible, while the projection function of an NF is invertible, resulting in XVAE & X = XSF (Kobyzev et al., 2021). Mcanwhile, GANs generally do not learn a map from the data space to the latent space, instead opting to only learn map from the latent space to the data space; finding or training the best methods for GAN in- version is an ongoing area of research (Karras et al., 2020; Xia al., 2022). Previous work on GAN inversion has shown that, without special attention to underrepresented data points during GAN training, reconstruction of certain data points can fail (Yu et al., 2020). As such, VAEs and NFs can perform local sampling of a data point by projecting perturbed version of its correspond- ing latent z back into the original data space, while GANs also have the potential to do so, albeit requiring suitable GAN inversion method (Wang et al., 2022; 7hu al., 2020). While the straight- forward tractability of VAE- or NF-based local sampling is attractive, GANs currently outperform VAES and NFs on popular tasks such as image synthesis (Zhang et al., 2022; Sauer et al., 2022). However, given the recent advent and popularity of diffusion models and the brittle nature of GAN training (Saxena & Cao, 2021), local sampling using diffusion models is promising avenue for leveraging local sampling techniques towards problems such as anonymization, data augmentation, and super-resolution.
We propose the Boomerang algorithm to enable local sampling of image manifolds using diffu- sion models. The algorithm earns its name from its principle mechanism- variance to push data away from the image manifold, and then using a diffusion model to pull the noised data back onto the manifold. The variance of the noise is the only parameter in the algorithm, and governs how similar the new image is to the old image, as reported by reported by Ho et al. (2020). We show how this technique can be used within three applications: (1) data anonymization for
noise of a certain
machine learning; (2) data augmentation; and (3) image super-resolution. We show that by exploiting the proposed local sampling technique we are able to anonymize the dataset and maintain better classification accuracy when compared with ofate-of-the-art generated data from
2

---

Under review as a conference paper at ICLR 2023
StyleGAN-XL (Saver et al., 2022). For data augmentation, we obtain higher classification accu- racy when trained on the show that Boomerang be used for super-resolution at varying strengths depending on the desired upsampling factor without the need to train different networks.
dataset versus no augmentation at all. Finally, we
In Section 2 discuss the training framework of diffusion models, introducing the forward and reverse processes. In Section 3 we introduce our proposed local sampling and provide insights on how the amount of added noise affects the locality of the resulting samples. Finally, we describe three applications (Sections to 6) that Boomerang can be used without any modification to the diffusion pretraining.
DIFFUSION MODELS
Generative models aim to sample from target probability distribution by using available samples from the distribution as training data, e.g., images of human faces. Diffusion models-a class of generative
this by learning reverse diffusion (forward) process, which involves adding Gaussian noise to the input image in steps (Ho et al., 2020). Given an image from the target distribution X p(xo), the forward process can be written as:
x v B,x,-1 Et, (,), t= 1=
where 3, E (0, f), f 1, ..... T. is the noise variance step t. which typically chosen beforehand (Song & Ermon, 2020). Since the transition from step t - 1 to is defined by Gaussian distribution 1, 3,I), the distribution of x; conditioned on the clean input image Xo can be expressed as Gaussian distribution,
the form of q(x,) xx-) == ((vi -,x,-1,
g(xx|x0) N (Va,xo, ot)[), t.....,,
with = Ii 3.). During training, diffusion models leam to reverse the forward process by starting at with a sample from the standard Gaussian distribution xT ~ NN(,,), I). The reverse process is defindd via Markov chain over Xo, X1,..., such that
X4-1 fx( mr. n. NO 3,I), t=
In the above expression, (x,, () is parameterized by a neural network with weights Φ, and 3,1 denotes the covariances step I. Equation (3) represents a chain with transition probabilities defined with Gaussian distributions with density
(x xx) == ( (),)),
1=
(4)
The covariance 3,1 in different steps can be also parameterized by neural networks, however, we follow Luhman & Luhman (2022) and choose 3. = 1- 8r, that matches the forward posterior distribution when conditioned on the input image gxx,1 x,, x,) (Ho et al., 2020).
1-a,
To ensure the Markov chain in Equation (3) reverses the forward process (Equation 1), the param- eters & are updated such that the resulting image at step / 0 via the reverse process represents sample from the target distribution p(xo). This be enforced by maximizing-with respect to & thehe likelihood (x)) of training samples where Xg represents the outcome of the reverse process at step t 0. Unfortunately, the density Ps (xx) does not permit a closed-form expression. Instead, given the Gaussian transition probabilities defined in Equation (4), the joint distribution over all the T + states can be factorized as,
T
Ps (x.......,
XT) = p(x)) II
Po (x; x:), (r(x))=N(0.))
(5)
t=1
with all the terms on the right-hand-side of the equality having closed-form expressions. To obtain a tractable expression for training diffusion models, we treat XI.... XT as latent variables and use the negative evidence lower bound (ELBO) expression for po(xo) the loss function,
T
Po (x xi)
((Φ)= = Ep(xo) (Eg(x....... xr...
logpr(Xx)) Σ 1=1
log
g(x, x,-1)
(6)
2 Ep(co) [-
3

---

Under review as a conference paper at ICLR 2023
After training, new global samples from (x() p(xo) are generated by running the reverse process in Equation (3) starting from Xy ~ (O, I). Due to the stochastic nature of the reverse process, particularly, the additive noise during each step, starting from two close initial noise vectors at step T does not necessarily lead to close-by images on the image manifold. The next section describes our proposed method for local sampling on the image manifold.
3 BOOMERANG METHOD
Our method, Boomerang, allows one to locally sample point X/ on an image manifold X' close to a point X X using a pretrained diffusion model fa. Since we are mainly interested in images, we suppose that Xo and x are images the image manifold X. We indicate how close to Xo want xg to be by setting the hyperparameter BBoomernng. We perform the forward process of the diffusion model BBommerang times, from = to = BBoomerang in Equation (1), and use fo to perform the reverse process from t = Booomerang back to t = O. Boommrrnng = we perform the full forward diffusion and hence lose all information about X,; this is equivalent to simply sampling from the diffusion model. We denote this partial forward and reverse process as [xn,, BBoomerngg) = x and call it Boomerang because X6 and x are close for small /noomerang. which can be seen in Figure 1.
When performing the forward process of Boomerang, it is not necessary to iteratively add noise BBoomernng times. Instead, we simply calculate the corresponding Oftammmus and sample from Equa- tion (2) once to avoid unnecessary computations. The reverse process must be done step by step however, which is where most of the computations take place, much like regular (non-local) sam- pling of diffusion models. Nonetheless, sampling with Boomerang has significantly lower computa- tional costs than regular sampling: the time required for Boomerang is approximately facumenne times the time for regular sampling. Moreover, we can use Boomerang to perform local sampling along with faster sampling schedules, e.g., sampling schedules that reduce sampling time by 90% (Kong & Ping, 2021) before Boomerang is applied. Pseudocode for the Boomerang algorithm is shown in Algorithm 1. SDEdit (Meng al., 2022) is an image editing method based pretrained diffusion models, which uses a similar algorithm as above. This method is specific to image editing and is not used in the context of local sampling. To the best of our knowledge, Boomerang is the first work which uses generative models for local sampling on image manifolds.
Algorithm Boomerang local sampling, given a diffusion model fa(x, t)
Input: Xo, Boooeeangg {at) To (3.)
Output: x
I0,
X monereonng 4y Xo + /1 Ot Boommnn E
for t = tBoomerang
do
if > then
8. -a
10
n (O, 3.I)
else
n
end if
x'_1 + (x(x), +n
end for return x6
We present a quantitative analysis to measure the variability of Boomerang Beomeraag-generatee images as Boomering is varied. As an expression of this variability, we consider the conditional distribution of samples generated through the Boomerang procedure conditioned on noisy input image at step tBoomerang, i.e., (x(1 X ). According to Bayes' rule, we relate this distribution to the distri- bution of noisy images at step Booomranng of the forward process,
Ps (x
X X X po (Xtrocnenng x6)x(x))
q( X /aaeeeerng x6)1(x6)
(7)
N (v Af X1x (1 - O' 1) p(x().
4

---

Under review as a conference paper at ICLR 2023
Class
0
2
4
5
6
7
8
Ground Truth
Boomerang
Figure 2:
Using Boomerang on CIFAR-10 to change the visual features of images. These images were created with a FastDPM Model.
The second line in the expression above assumes that by training the diffusion model via the loss function in Equation (6), the model will be able to reverse the diffusion process at each step of the process. The last line in the equation above, which follows from Equation (2), suggests that the density of Boomerang-geneet
images is proportional to the density of normal distribution with covariance (1 Of I times the clean image density p(x(), which the same as the original clean image density. The resulting density will have very small values far away from the mean of the normal distribution. In addition, the high probability region of (x X(g grows as 1 Anonmeang becomes larger. This quantity monotonically increases Boommcrang goes from one to since a = I (1 3,) and B E (0, 1). As a result, we expect the variability in Boomerang- Boomerang for larger Booomeang steps.
generated images to increase
Since Boomerang depends on a pretrained diffusion model fa. it does not require the user to have access to significant amount of computational resources data. This makes Boomerang very ac- cessible to practitioners and even everyday users who don't 't have specialized datasets to train a diffusion model for their specific problem; they just need to find diffusion model which is trained on diverse enough images, such as Stable Diffusion (Rombach et al., 2022). The main limitation of using Boomerang, however, is that the practitioner must find a diffusion model which models the image manifold well. If fo does not generate realistic images, then the output of Boomerang will also suffer, as we have seen empirically. However, this is becoming less and less of problem with the advancement of diffusion models in image synthesis tasks. Overall, our Boomerang method allows local sampling on image manifolds without requiring significant amounts of computational resources or data.
APPLICATION 1: ANONYMIZATION OF DATA
4.1 DATA AND MODELS
In the following experiments 2019), and ILSVRC2012 (ImageNet) (Russakovsky et al., 2015) datasets. For the ImageNet ex- periments, we use 200-class subset of ImageNet which call ImageNet-200; these are the 200 classes that correspond to Tiny ImageNet (Russakovsky et al., 2015). We use Boomerang with the Stable Diffusion (Rombach al., 2022), Patched Diffusion (Luhman & Luhman, 2022), and FastDPM (Kong& & Ping, 2021)² models, with some comparisons to the recent StyleGAN-XL (Sauer al., 2022). For the FastDPM model we use Boomerang on the STEP DDPM sampler with S 100 steps out of the original 1000 steps.
the CIFAR- 10 (Krizhevsky et al., 2009), FFHQ (Karras et al.,
4.2 ANONYMIZATION
Recent work has proved that, not only are overparameterized models (such as deep networks) vulner- able to membership inference attacks (Shokri et al., 2017), but also that their vulnerability increases as their number of parameters increases (Tan et al., 2022). Such attacks attempt to recover poten- tially sensitive training data (e.g.. medical data, financial information. and private images) given only access to a machine learning model (e.g., classifier) trained on that data. Local sampling
We use this repository.
We use this repository.
5

---

Under review as a conference paper at ICLR 2023
Class
0
32
67
142
183
Ground Truth
Boomerang
Figure 3: Using Boomerang on ImageNet-2000 to change the visual features of images. These images were created with a Patched Diffusion Model.
presents potential tool towards privacy-preserving machine learning by enabling the generation of data points that are similar to real (i.e., sensitive) data, while being dissimilar enough to existing data so not provide any true sensitive information, in the case of membership inference attack. The Boomerang algorithm not only enables friendlyy local sampling for anonymization, but also enables the use of praited and popular diffusion models in doing so, such Stable Diffusion.
Stable Diffusion (Rombach et al., 2022) is a promising venue for Boomerang image anonymization, capable of both image feature modification and image style transfer, having been trained on vast corpus of images containing wide variety of images and styles (Schuhmann et al., 2021). Unlike other diffusion models, Stable Diffusion uses a Latent Diffusion Model (LDM) to conduct all for- ward and reverse diffusion processes solely in latent space. As such, the Boomerang algorithm, when used on Stable Diffusion, first projects an image into the latent space using the LDM encoder, then adds noise to the latent vectors, next iteratively denoises the latent vectors via the LDM verse process, and finally re- re-projects the denoised latent vectors back into the image space using the LDM decoder. Figure 1 (and Figures 5 to 7 in Appendix A. showcase the ability of Boomerang and Stable Diffusion to produce not only similar (i.e., anonymous) variants of an initial image, but also dissimilar variants that share certain features (e.g., cat resembling person, or bathroom resembling a bedroom).
Additionally, Boomerang can be used make entire datasets anonymized to varying degrees con- trolled by the hyperparameter Boomenang Anonymization with people's 's faces, for example, makes intuitive sense, but anonymized images in a more general context, such as ImageNet, is less clearly defined. In this work, an image Xn is anonymized to X² if the features of cach image are visibly different so that an observer would guess that the two images are of different objects. For each diffusion model, we pick Boomerang so that the anonymized images are different, but not drastically different from the original dataset images, for CIFAR- 10 and ImageNet-200. These images are shown in Figures 2 and 3. One potential alternative to using completely synthetic data. The state-of of-the--rtt generative model for image generation of both Im- ageNet and CIFAR- 10 is StyleGAN-XL. We compare our anonymized data to training classifier with completely synthetic data.
data is to use
We show in Table that training classifier on these anonymized images achieves performance be- tween using real images and using the state-of-the-art ImageNet generated images from StyleGAN- XL. This makes sense because, we increase Boomcrang. we obtain samples that are more similar to completely synthetic images. Therefore, anonymization of datasets is better alternative for preserving the identity of data than using generated data.
5 APPLICATION 2: DATA AUGMENTATION
Data augmentation for image classification essentially done by sampling points on the image manifold A' near the training data. There typical augmentation techniques, such using random

---

Under review as a conference paper at ICLR 2023
Table 1: Using boomerang-generaine data for data augmentation increases accuracy of CIFAR- 10 and ImageNet-200 classification tasks. Using just the "anonymized" boomerang dataset for classification performs between using real data and using state-of-the-ar completely synthetic StyleGAN-XL data.
Classification
Top-1 I Test Accuracy
Top-5 Test
Training data
Task
Accuracy
CIFAR-10
CIFAR-10 data
87.8%
CIFAR-10
StyleGAN-XL generated data
81.5%
CIFAR-10
Boomerang (anonymized) data (ours)
84.4%
CIFAR-10
CIFAR-10 + Boomerang DA (ours)
88.4%
ImageNet-200
ImageNet-200 data
66.6%
85.6%
ImageNet-200
StyleGAN-XL generated data
50.2%
73.0%
ImageNet-200
Boomerang (anonymized) data (ours)
61.8%
83.4%
ImageNet-200
ImageNet-200 + Boomerang DA (ours)
70.5%
88.3%
image flips and crops, which exploit symmetry and translation invariance properties of images. Al- though there many techniques for data augmentation, they mostly involve modifying the training data in ways which make the new data resemble the original data while still being different, i.e., they attempt to perform local sampling the image manifold.
Due to the intrinsic computational costs of diffusion models, we generate the augmented data before training instead of on-the-fly generation during training. We pick Boooerang be large enough to produce differences between the original dataset and the Boomerang gerrraattgg dataset, as shown in Figures and 3. We then randomly choose to use the training data or the Boomerang gennnrted data with probability 0.5 at each epoch. We use ResNet- 18 (He et al., 2016) for our experiments.
We show that using Boomerang data augmentation on CIFAR- 10 classification increases test accu- racy from 87.8% to 88.4%. For CIFAR-10 we pick BBomeerngg = 40 and train for 100 epochs with a multistage learning rate scheduler which reduces the learning rate by factor of 10 epochs 30, 60, and 80. We train using stochastic gradient descent with an initial learning rate of 0.1, Nesterov momentum of 0.9, and weight decay of 5e-4. Since CIFAR-10 10 already has significant amount of samples per class (5,000)
a small benefit from using Boomerang data augmentation.
We also perform data augmentation with ImageNet-2000 described in Section 4.1, raising our test accuracy from 66.6% to 70.5%. We pick fBoomerang 75 and train for 90 epochs using the standard PyTorch (Paszke et al., 2019) training script* with few modifications. We removed the standard data augmentation of random resized cropping and random flips and instead resize each image so that its smallest dimension is 256 followed by centered crop to 224 224 in order to see if our Boomerang data augmentation would be beneficial. Since ImageNet-200 has much fewer samples than CIFAR-10 per class (approximately 1,300) and the classification task is harder, the greater increase in performance due to our data augmentation makes sense.
APPLICATION 3: SUPER-RESOLUTION
6.1
VANILLA BOOMERANG
To perform super-resolution with Boomerang, one first upsamples low-dimensional image using any existing method (e.g., nearest-neighhor, linear interpolation). and then performs the Boomerang algorithm (See Algorithm I) on this image to recover the "original" image in high dimensional space. The recovered image corresponds to a point on the image manifold that is "close" to the noisy, upsampled image; the distance between the two images is controlled by Boomerang
While others such as Saharia et al. (2022b) and Rombach et al. (2022) have used diffusion models to perform well super-resolution tasks, Boomerang has two key advantages to vanilla diffusion
We use code from this repository to implement the CIFAR-10 experiments. Forr 2m0 training we used the standard PyTorch training script.

---

Under review as a conference paper at ICLR 2023
Initial 8x DS Image
ncascade
Teascude 10
Ground Truth
Figure 4: Cascaded Boomerang super-resolution; Boooerang 50
models. The first is that Boomerang's local sampling keeps the output "close" to the input on the image manifold. The second is that adjusting Boomerang allows easy "tuning" of how aggressively the super-resolution enhancement is applied. This means that the same pretrained network can perform super-resolution on images different scales; the more downsampled or degregated the image, the larger the value of Boumenang one may choose to use to fill in more details of the final image. Fur- thermore, by varying Booomerang for different passes of the same input image, one can choose which image has the best detail/variance tradeoff. This can be seen in Figure 8 in Appendix A.I, wherein more aggressive super-resolution improves clarity at the cost distance from the original image. Empirical tests showed that setting tBoomerang z 100 on the Patched Diffusion Model produced good balance between sharpness and the features the original image, in some cases maximizing PSNR.
6.2 CASCADED BOOMERANG SUPER-RESOLUTION
As Boomerang is increased, the variance of the generated images dramatically increases due to diffu- sion models' stochastic nature (noise is added to data at each step Algorithm 1). As result,
8

---

Under review as a conference paper at ICLR 2023
increasing BBoomerang enough causes the generated images to vary so much that they no longer semble the imput image at all (and thus we are not sampling from the image manifold as closely as desire). Even for modest values of Boomerang the large variance of added noise causes repeated upsampling attempts to differ significantly. In this section, we propose a simple method that allows the results of different upsampling attempts be stabilized.
The cascade method describes repeated passes of an upsampled image through a diffusion network with a smaller value of (Roomerang- If we denote the Boomerang upsampling method in the previ- ous section on an input image Xds to generate Xar = Big (Xas), the method described here would be Xcascade- B [By, (- (... (By, )))). We designate Reaseade as the number of times we repeat Boomerang on the intermediate result. In addition stabilizing independent super-resolution at- tempts, the cascade method allows users to iteratively choose the desired upsampling detail- simaly stop repeating the cascade process once the desired resolution is achieved. An example of cascaded super-resolution with Boomerang is shown in Figure 4, with more details found in Figure 9.
Super-resolution an essential inverse image problem (Freeman et al., 2002; Wang al., 2021), and Boomerang is a quick and efficient method of performing super-resolution with diffusion networks. As we have shown, Boomerang allows the user to easily adjust the strength of the super-resolution by varying /Boomerang. The same procedure and pretrained network can thus be used to perform super-resolution any scale, for any image size smaller than the output of the diffusion network (in our emperical tests, the image dimensions were 1024 x 1024): for larger scaling factors, simply increase Boomerang or cascade the result until one achieves the desired fidelity. This also avoids the issue of needing train different networks for upsampling different scales. As with the previous applications, Boomerang's 's generated images look realistic yet it requires no additional fine-tuning or training.
Interestingly. performing super-resolution using Boomerang with some diffusion models worked and with some it did not work. With Stable Diffusion, for example, empirical results had shown that the vanilla Stable Diffusion network would not introduce detail into blurred images except for large Boomerang which point the generated images no longer strongly resembled the ground truth images. This is likely due to the fact that noise is added in the latent space with Stable Diffusion but in the image space with Patched Diffusion. Therefore, the noise model will impact which kinds of inverse problems Boomerang will be effective on.
7 CONCLUSION
We have presented the Boomerang algorithm, which enables simple and efficient local image sam- pling via diffusion models on single GPU, without any re-training or modifications the model. We showed the applicability of Boomerang on various tasks, such as anonymization, data augmen- tation, and super-resolution. Future works include continued experiments of its efficacy for data augmentation, as well applying the Boomerang algorithm to different domains of data, such as audio and text. Additionally, recent work has shown that diffusion models can work with non- stochastic transforms instead of additive Gaussian noise (Bansal et al., 2022; Daras et al., 2022), and evaluating the Boomerang algorithms with such diffusion models would provide further insight into the nature of local sampling using diffusion models.
ACKNOWLEDGEMENT
This work was supported by NSF grants CCF-1911094, IIS-1838177, and IIS- ONR grants N00014-18- 18-115511,,, N00014-20-15 20-5544,,, and MURI N00014-20-1-27877, AFOSR grant FA9550-22- 1-0060; and Vannevar Bush Faculty Fellowship, ONR grant N00014-18-1-2047.
REFERENCES
Arpit Bansal, Eitan Borgnia, Mong-Min Chu, Jie S Li, Hamid Kazemi, Furong Huang. Micah Gold- blum, Jonas Geiping, and Tom Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise. arXiv preprint arXiv:2208.09399, 2022.

---

Under review as a conference paper at ICLR 2023
Sam Bond-Taylor, Adam Leach, Yang Long, and Chris G. Willcocks. Deep generative modelling: A comparative review of VAEs, GANs, normalizing flows, energy-based based and autoregressive models. IEEE Transactions Pattern Analysis and Machine Intelligence, pp. 1-1, 2021. doi: 10.1109/ TPAMM.2021.3 16668.
Giannis Daras, Mauricio Delbracio, Hossein Talebi, Alexandros G Dimakis, and Peyman Milanfar. Soft diffusion: Score matching for general corruptions. arXiv preprini urXiv:2209.05442 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 1780-8794, 2021.
Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. A survey of data augmentation approaches for NLP. In Findings the Association for Computational Linguistics: ACL-IJCNLP 2021, pp.
2021.
W.T. Freeman, T.R. Jones, and E.C. Pasztor. Example- based super-resolution IEEE Computer Graphics and Applications, 22(2):56-66 65, 2002. doi: 10.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. In Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Advances in Neural Infor- mation Processing Systems, volume 27, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pp. 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative the JEEE/CVF conference on computer vision and pattern
adversarial networks. In Proceedings the recognition, pp. 4401-4410, 2019.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyz- ing and improving the image quality of StyleGAN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Bahjat Kawar, Gregory Vaksman, and Michael Elad. Stochastic image denoising by sampling from the posterior distribution. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1 1866-18755 2021.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. International Conference on Learning Representations, 2014. doi: 1048550ARKXX.131...14.
Ivan Kobyzev, Simon J.D. Prince, and Marcus A. Brubaker. Normalizing Flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):3964-399) 2021. doi: (C09/TPAM..2020.2999777
Zhifeng Kong and Wei Ping. On fast sampling of diffusion probabilistic models. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
Troy Luhman and Eric Luhman. Improving diffusion model efficiency through patching. arXiv preprint arXiv:2207.00 04316, 2022.
Chenlin Meng, Yutong He, Yang Song. Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In Interna- tional Conference on Learning Representations, 2022. URL https: http://pppeeeeeeeeeeeeee net forum? d=andddaaaaacPtt.....
10

---

Under review as a conference paper at ICLR 2023
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for GANs do actually converge? Proceedings of the 35th International Conference on Machine Learning, volume 80, pp. 3481-3490, 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library, 2019.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06/25, 2022.
Danilo Rezende and Shakir Mohamed. Variational Inference with Normalizing Flows. In Fran- cis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37, pp. 1530-1538, 1538, 2015.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3)211-2522 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205. 2022a.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-14, 2022b. doi: 10.1 009//TPAM..2022.3244442
Axel Sauer, Katja Schwarz, and Andreas Geiger. StyleGAN-XL: Scaling StyleGAN to large diverse datasets. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-10, 2022.
Divya Saxena and Jiannong Cao. Generative Adversarial Networks (GANs): Challenges, solutions, and future directions. ACM Comput. Surv., 54(3), 2021. ISSN 0360-0300. doi: 10.1 10.1444/3446774
Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarezyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:21 2021.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE Symposium Security and Privacy (SP), pp. 3-18, 2017. doi: 10.1 10.1109/SP.211..41
Yang Song and Stefano Ermon. Improved techniques for training score- baooe- generative models. In Advances in Neural Information Processing Systems, volume 33, pp. 12438-12448, 2020.
Jasper Tan, Blake Mason, Hamid Javadi, and Richard G Baraniuk. Parameters or privacy: A provable tradeoff between overparameterization and membership inference. arXiv preprins 200.022202.01243. 2022.
Tengfei Wang. Yong Zhang, Yanbo Fan, Jue Wang, and Qifeng Chen. High-fidelity GAN inversion for image attribute editing. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 11379-11388 2022. doi: 10.45550/ARXIV.2109..66
Zhihao Wang, Jian Chen, and Steven C. H. Hoi. Deep learning for image super-resolution. A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3365-3333 3387, 2021. doi: 10.1109TTPA MI. M0.000 2982166.
11

---

Under review as a conference paper at ICLR 2023
Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, and Huan Xu. Time series data augmentation for deep learning: A survey. In Proceedings of the Thirtieth International Joint Conference Artificial Intelligence, IJCAI-21, pp. 4653-4660) 2021. doi: 24449663/ccai2222/3333
Sebastien C. Wong, Adam Gatt, Victor Stamatescu, and Mark D. McDonnell. Understanding data augmentation for classification: When warp? In 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1-6, 2016. doi: 10.1109/DICTA. 199707797091.
Weihao Xia, Yulun Zhang, Yujuu Yang, Jing-Hao Xue, Bolei Zhou, and Ming-Hsuan Yang. GAN inversion: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
Abhay Yadav, Sohil Shah, Zheng Xu, David Jacobs, and Tom Goldstein. Stabilizing adversarial nets with prediction methods. International Conference on Learning Representations, 2018.
Suorong Yang, Weikang Xiao, Mengcheng Zhang, Suhan Guo, Jian Zhao, and Furao Shen. Image data augmentation for deep learning: A survey. Journal of Big Data, 2022. doi: 10. 10.1186//40033--119-999-99
Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, and Mario Fritz. Inclusive GAN: Im- proving data and minority coverage in generative models. In European Conference on Computer Vision (ECCV), 2020. doi: 01177998833030-584444
Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo. StyleSwin: Transformer-base GAN for high-resolution image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1334-11314, 2022. doi: 10.4850/ARXV.2112..000
Jiapeng Zhu, Yujun Shen, Deli Zhao, and Bolei Zhou. In-domain GAN inversion for real image editing. In European Conference on Computer Vision (ECCV), 2020. doi: 10.48550/ARXIV. 2004.00049.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A. BOOMERANG-GENNRATEED IMAGES VIA THE STABLE DIFFUSION MODEL.
Here present additional images created via the Boomerang method that indicate the evolution of the predicted image as we increase Booomerang. These images are generated via the pretrained Stable Diffusion model (Rombach al., 2022) where instead of adding noise the image space during the forward process it is added in the latent space. Figures 5-7 showcase this where the images on bottom row show noisy latent variables whereas the ones on the top row indicate the Boomerang predictions with increasing amounts of added noise from left to right, except for the rightmost image, which is created by using an alternate prompt.
A.2 VANILLA BOOMERANG SUPER RESEER-REOOUIOO
Here present the result of image super-resolution using the vanilla Boomerang approach. Fig- ure 8 illustrates the results for image super-resolution. The top-left image in this figure shows the low-resolution image, and the top-right and bottom-lefi images are the result of vanilla Boomerang 8x super-resolution when using Boomerang = 100 and Beomerang = 150, respectively. When com- pared with the high-resolution image in the bottom-right corner of Figure we observe that the resulting image with Boomerang 100 is plausible while the result with Boomerang 150 seems high-resolution, but is inconsistent compared to the original image.
13

---

Under review as a conference paper at ICLR 2023
Back prompt "cat"
Back 0 prompt "cat"
Back 0 prompt "person"
Back prompt "cat"
Back prompt "person" "
Initial Image
1 Forward to 800
Forward to 500
Forward 900
Figure 5: The Boomerang method using Stable Diffusion (T 1000), as in Figure 1, with an image of a cat. Note how, as fasuense approaches 1, the content of Goomerang-generaan images strays further away from the starting image.
Back to t = 0 prompt "bedroom"
Back to t = 0 prompt "bedroom"
Back to = 0 prompt "bedroom"
Back to t = prompt "bathroom"
Initial Image
ue
Forward to t 200
Forward to t 500
Forward to t = 800
Figure 6: The Boomerang method using Stable Diffusion, as in Figure 1, with an image of a bed- room.
14

---

Under review as a conference paper at ICLR 2023
Back to f prompt "person"
Back /. prompt "person" "
Back = 0
Back = 0 prompt "dog"
Initial Image
prompt "person"
Forward to 200
Forward to 500
Forward to 800
Figure 7: The Boomerang method using Stable Diffusion, in Figure 1, with an image of Albert Einstein.
15

---

Under review as a conference paper at ICLR 2023
Initial 8x DS Image
Boomerang 100
Boomerang 150
Ground Truth
Figure 8: Vanilla super-resolution with Boomerang.
16

---

Under review as a conference paper at ICLR 2023
Dess 6
Nesse
Ground Truth
Dcasc 6
Tlassc 8
Ground Truth
Figure 9: Cascaded boomerang examples with superresolution. For the top row, notice how the best quality image is seen after cascade steps, with BBooeerng = 50. After cascades, details such as the teeth begin to be removed. On the bottom row, however, Plaase 8 provides better results and more detail compared to previous steps. This shows the value in introducing the cascade method: different images may have better results with more cascade steps than others
17


Reviews:
[{"review_id": "m3FroWkqG6", "reviewer": "Reviewer_Wv4p", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method for generating variations of an existing image with diffusion models. The key idea is to add noise to the given image and then reconstruct it using denoising diffusion models. As you add more noise, the variability increases.", "summary_of_the_review": "The paper introduces a method for producing variations of a given image. I believe that this method is already known. Hence, I recommend rejection.", "strength_and_weaknesses": "I think what this paper proposes is known and not new. Particularly, it seems that the Boomerang Algorithm is exactly the same as the SDEdit paper: https://arxiv.org/abs/2108.01073. The latter is not cited and the differences are not discussed, but I really don't understand if there is anything new here.\n\nThe super-resolution application seems interesting. However, it would be good to see MSE and FID scores and compare it against other approaches - e.g. compare with super-resolution with GANs.\n\nFor the data augmentation application, I think that the comparison with StyleGAN-XL is unfair. Unconditional generation with StyleGAN-XL is expected to not match the quality of the original data samples. But data augmentation with Boomerang is not unconditional. Dataset samples are only slightly augmented. A more fair comparison would be to do inversion with StyleGAN-XL and add the inverted images to the dataset. There are many methods for controlling the fidelity of the inverted image to the given input which would be the equivalent of the t_boomerang knob. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "ex3cd6gETHi", "reviewer": "Reviewer_z9Wv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This submission proposes a method for inverting samples in diffusion generative models. The proposed approached simply applies the forward diffusion process for a given number of steps and then samples from the resulting latent. The approach is evaluated on 3 different problems: privacy-preserving reconstruction, data augmentation and super-resolution. ", "summary_of_the_review": "This submission tackles an interesting problem (eg. Inversion in diffusion models). The paper is clearly written and easy to understand. The main drawback of the proposed approach is the novelty of it, since an equivalent method was already introduced in the original DDPM paper.  Furthermore, the qualitative evaluation is not complete and makes it hard to validate the overall value of the contribution", "strength_and_weaknesses": "Strengths:\n- Successfully inverting a sample in diffusion models is an interesting problem that deserves more attention from the community.\n- The proposed approach is technically correct.\n\nWeakness:\n- The technical contributions are not novel, the same approach was introduced in the original DDPM paper https://arxiv.org/abs/2006.11239  (see appendix for interpolation results).\n- The empirical evaluation is rather limited. It is hard to evaluate the value of the contribution with qualitative results on data augmentation. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "Yt1eEuMOOa", "reviewer": "Reviewer_ydwA", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce a method for local sampling in diffusion models. This is achieved by leveraging the stochasticity of the diffusion models. An input image is propagated through the forward process for t steps then through the reverse process for t steps. In the forward process each step produces an output that is the linear combination of the input and a noise vector – the combination coefficient for the input is determined by the choice of variance for the noise. The reverse process is governed by a learned transition function. The authors demonstrate the utility of their local sampling procedure for anonymization of data, dataset augmentation, and image super-resolution.", "summary_of_the_review": "I think the authors have identified a simple and powerful approach for local sampling in diffusion models. The paper is clearly written, and appears detailed enough that others could reproduce the findings.", "strength_and_weaknesses": "*Strengths:* This paper introduces a method for locally sampling the data manifold in a diffusion model. The proposed method is intuitive and the authors have demonstrated the applicability of the approach to several tasks. Moreover, the method proposed can be used with pretrained models, on a single GPU making it accessible to the larger community. \n\n*Possible typos:*\n- “better alternative to preserving” → better alternative for preserving\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PjTcUHto8d", "reviewer": "Reviewer_qWcF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper under consideration proposes a “Boomerang” approach for image variability-related applications. The idea is quite simple. Given a standard diffusion model:\n- run forward diffusion for several times resulting in noised image\n- run backward diffusion for several times resulting in an image which is controllably different from the original one.\n\nThe authors demonstrate the application of the aforementioned method in several applications.\n", "summary_of_the_review": "Summary Of The Review: The proposed approach is quite straightforward and incremental. The applications are nice and practically interesting but require further investigation. \n", "strength_and_weaknesses": "Strength: The idea is quite simple and natural and could be applied for arbitrary pretrained diffusion models.\n\nWeaknesses: \nIn general. My main concern about the paper is that the proposed approach is too incremental and kind of obvious. It is really nice that such a simple method helps to solve rather important problems listed in the application sections but from my point of view it is not enough reason for the paper under consideration to be published in top A* conference.\n\nIn particular. \n1) No comparison with competitive super-resolution methods is provided. Therefore it is difficult to estimate the usefulness of the Boomerang approach in this application. \n2) No quantitative comparison of the vanilla Boomerang and cascaded Boomerang super-resolution is provided. It is not clear, if the “casacadeness” actually improves samples quality\n3) Table 1 doesn’t seem to be convincing and requires clarifications. The reported accuracies of ResNet-18 model trained on standard Cifar-10 (and, probably ImageNet-200) data are actually not sota accuracies for ResNet-18 and corresponding datasets (see, https://github.com/kuangliu/pytorch-cifar for example). Therefore they are just particular accuracies obtained in particular training loops. This observation gives rise to the questions like as follows: Can we just train ResNet-18 for 200 (180 for ImageNet) epochs and achieve the competitive accuracy compared to Boomerang-augmented dataset cases? Can we achieve competitive performance using standard augmentation techniques? \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject