Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Authors: Soobin Um
Beomsu Kim
Jong Chul Ye
Venue: ICML 2025 poster (2025)
Keywords: diffusion models
minority sample generation
image generation
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: We develop a simple guidance-free approach for minority sample generation using diffusion models.

Abstract: Minority samples are underrepresented instances located in low-density regions of a data manifold, and are valuable in many generative AI applications, such as data augmentation, creative content generation, etc. Unfortunately, existing diffusion-based minority generators often rely on computationally expensive guidance dedicated for minority generation. To address this, here we present a simple yet powerful guidance-free approach called *Boost-and-Skip* for generating minority samples using diffusion models. The key advantage of our framework requires only two minimal changes to standard generative processes: (i) variance-boosted initialization and (ii) timestep skipping. We highlight that these seemingly-trivial modifications are supported by solid theoretical and empirical evidence, thereby effectively promoting emergence of underrepresented minority features. Our comprehensive experiments demonstrate that Boost-and-Skip greatly enhances the capability of generating minority samples, even rivaling guidance-based state-of-the-art approaches while requiring significantly fewer computations. Code is available at https://github.com/soobin-um/BnS.

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Soobin Um
Beomsu Kim
Jong Chul Ye
Abstract
20
EDM
Prev. Pareto Frontier
Minority samples are underrepresented instances located in low-density regions of data mani- fold, and are valuable in many generative Al ap- plications, such as data augmentation, creative content generation, etc. Unfortunately, existing diffusion-basee minority generators often rely on computationally expensive guidance dedicated for minority generation. To address this, here we present a simple yet powerful guidance-free ap- proach called Boost-and- -Sds-- for generating mi- nority samples using diffusion models. The key advantage of our framework requires only two minimal changes to standard generative processes: (i) variance-boostee initialization and (ii) timestep skipping. We highlight that these seemingly- trivial modifications are supported by solid the- oretical and empirical evidence, thereby effec- tively promoting emergence of underrepresented minority features. Our comprehensive experi- ments demonstrate that Boost-and-Skip greatly enhances the capability of generating minority samples, even rivaling guidance-based state-of- the-art approaches while requiring significantly fewer computations. Code is available at https: / //github... com/ wttp:/oooonnooooo wuuuumm/ooo BBnS.
ADM
18
Temperature sampling
16.
14
Um et al. (2023)
12
Boost-and-Skip (ours)
Schwag et al. (2022)
10
Um and Yo (2024)
1.0
2.0
2.5
3.5
Inference time (sec/ / sample) 4
Figure 1: Trade-off between minority generation perfor- mance and complexity on ImageNet 64 x 64. The area of each bubble corresponds to peak memory consumption. cFID (Parmar et al., 2022) calculations are based on real minority data (as in Um al. (2023); Um & Ye (2024b)), meaning that lower values indicate generation of more real- istic minority samples. Note that our framework achieves competitive minority generation performance comparable to guided samplers (Sehwag et al., 2022; Um et al., 2023; Um & Ye, 2024b) while maintaining minimal complexity, thus advancing beyond the existing Pareto Frontier. See Table for detailed metric values.
1. Introduction
has been readily adopted by researchers, leading to signifi- cant progress in minority generation the task of generating minority samples that reside in low-density regions of data manifold! (Sehwag et al., 2022; Um et al., 2023; Um & Ye, 2024b;a). The distinctive characteristics of minor- ity instances make them important in various applications, including medical diagnosis (Um al., 2023), anomaly de- tection (Du al., 2022; 2023), and creative AI (Rombach et al., 2022; Han et al., 2022).
Diffusion models are a prominent class of modern gener- ative AI, known for their ability to generate high-quality content across various data modalities (Ho et al., 2020; 2022; Zhang al., 2023). Unlike traditional frameworks like GANs (Goodfellow al., 2014), diffusion models are notably robust, effectively learning the underlying data dis- tribution even for rare or underrepresented examples in the training data (Sehwag et al., 2022). This inherent advantage
'Equal contribution 'Graduate School of AI. Korea Ad- vanced Institute of Science and Technology (KAIST), Dae- jeon, Republic of Korea. Correspondence Jong Chul Ye <jong.ye ye kr.>..
Existing high- sigghperffrmance minority generators primarily rely upon guided sampling (Sehwag et al., 2022; Um et al., 2023; Um & Ye, 2024b;a). For instance, the authors in Se-
FFrmally, the task of minority generation is expressible as drawing instances from S. = {z E M Pe po(x) < <e), where M represents the data manifold, and po denotes the implicit density learned by the diffusion model. Here e small positive constant.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
hwag al. (2022); Um et al. (2023) employ classifier guid- ance (Dhariwal & Nichol, 2021) to steer generation toward low-density regions using separately trained classifiers. A more recent approach by Um & Ye (2024b) mitigates this reliance on external components (such as classifiers) by de- veloping a self-contained sampler that derives low-density guidance solely from a pretrained diffusion model. How- ever, this method incurs significant computational overhead due its reliance on backpropagation through the diffu- sion network to compute the guidance term. Although Um & Ye (2024a) introduced a refined guidance term, specifi- cally for text-to-image generation (Nichol et al., 2021), the computational bottleneck persists.
2024b), while offering substantially reduced computational costs. For instance, on ImageNet 64 x 64, our minority sampler requires 65% less wall-clock time and 4.5 times lower peak memory consumption than the current of the-art method by Um & Ye (2024b). Moreover, thanks to the simplicity, our method is highly scalable. See Figure I for a visual illustration of these benefits and Table for detailed metric values. To further highlight the practical significance of our sampler, we demonstrate its effectiveness in a potential downstream application, specifically its use in data augmentation for classification tasks.
Our contributions are summarized as follows:
We propose Boost-and-Skip a novel guidance-free ap- proach that introduces two simple yet effective modifi- cations standard stochastic generative processes: (i) variance-boosted initialization and (ii) timestep skip- ping.
In this work, we depart from the paradigm of guided sam- pling and develop a simple and computationally efficient technique for minority generation. To this end, we first in- vestigate potential approaches for guidance-free minority sampling and highlight their pitfalls. We then present Boost- and-Skip, our novel guidance-free approach that sidesteps the previous issues. At a high level, our framework en- ables minority-favorell initialization that encourages emer- gence of underrepresented features throughout generative processes without the help of additional low-density guid- ance. Boost-and-Skee Skip accomplishes this through two simple modifications to standard stochastic generative processes like DDPM (Ho et al., 2020).
We provide both theoretical insights and empirical evi- dence to validate the effectiveness of the two modifi- cations, which together lead to significantly improved minority generation performance.
We empirically show that our approach achieves com- petitive performance with substantially lower compu- tational costs compared to state-of-the-ar methods.
2. Background
Specifically, we propose to initiate stochastic generation with variance-boosted noise instead of standard Gaussian noise to encourage initializations in low-density regions. Our second yet critical modification involves skipping sev- eral of the earliest timesteps to enhance the impact of low- density initialization from the first modification. We demon- strate, both theoretically and empirically, that these two modifications together lead to significantly improved minor- ity generation, whereas each modification alone only yields marginal gains. In particular. we invoke stochastic contrac- tion theory (Pham, 2008; Pham et al., 2009; Chung al., 2022) to show that the contracting nature of the stochastic generative process gradually corrects unwanted spurious information introduced by the noise amplification, thereby contributing to the generation of high-quality minority sam- ples.
Diffusion models are characterized by the two stochas- tic differential equations (SDEs): the forward and reverse SDEs (Song al., 2020). Given d-dimensional noise- perturbed data space æ+t) E gd with & E [0, T). the forward SDE is conventionally expressed as:
da: = f(x ()dl + g())dur
where f(.-): R2 + Rd is drift term, and g(-) : R + R indicates (scalar) diffusion coefficient coupled with standard Wiener process
Rd. With properly chosen
f(;)) and g(-), the forward SDE describes a progressive destruction of the target data distribution x00) ~ Po upto a fully noised one xT) ~ pr that one can easily sample from (e.g.. a Gaussian distribution).
To demonstrate the empirical benefits of our approach, we conducted extensive experiments across various real-world benchmarks. Importantly, our sampling technique achieves competitive minority generation performance compared to prior works (Sehwag et al., 2022; Um et al., 2023; Um & Ye,
The reverse SDE runs backward from the noised distribu- tion x(T) ~ pr down to the data distribution z00 Po- formally written as (Song et al., 2020):
dæ [(x ((() v log di + g(()dur, (2)
"By guidance-free, we refer to approaches that do not incorpo- rate dedicated guidance term specifically designed promote minority sample generation. Note that this does not preclude the use of conventional guidance mechanisms, such as classifier-fie free guidance (Ho& & Salimans, 2022).
where w is a standard Wiener process with backward time flow (from T to 0). The score function V, log is commonly approximated by a neural network sa t) via denoising score matching (Vincent, 201 1; Song et al., 2020).
One prominent instance of diffusion processes is variance-
2

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
O
(a) Ground truth
(b) Diffusion
(c) Temperature sampling
(d) Boost only
(c) w/o SDE
(i) B&S (Ours)
Figure 2: A 2D distribution with two concentric circles indicated by blue and red. Red circle represents minority samples, being sampled x 10 less compared to the blue circle. (a) Ground truth samples. (b) Standard diffusion sampling assigns even less mass to minority samples. (c) Temperature sampling (Ackley et al., 1985) generates off-manifold samples, due to errors accumulated through sampling trajectories (see Figure 3 for details). (d) Boosting without skipping closely recovers the data distribution yet does not promote minority features over the ground truth. (e) Without stochasticity (e.g., using PF-ODE), the proposed modifications fail to contract truncation error, leading to off-manifold samples. (f) Boost-and-Skip (B&S) generates minority samples without falling off the data manifold.
preserving (VP) SDE (Song al., 2020):
By discretizing Eq. (2), the associated reverse process can be written as (Song et al., 2020):
dæ 1 3(1) + v B())dw.
x + (1 a.) log + vI a;z,
2;-1
Vaa
where 3(t) is a positive function which integrates to infinity to ensure pr xe (N, VP-SDE conditioned on an ini- tial point (0) can be characterized by Gaussian forward diffusion kernel (Song et al., 2020):
(8)
where E (I...., and V (o) represents the score function for discrete timesteps. Data generation is performed by first sampling from æ ~ ~ I) and simulating Eq. (8) down to Zo with the learned score net- work 8p (x, i) Rs Va (x)).
(z = =((x(())(()))()) (z)) (4)
where (t) is defined as
3. Method
(() e à So 8(x)as
3.1. Towards guidance-free minority sampler
The associated reverse process of VP-SDE reads:
Our framework starts by investigating potential approaches minority-focused generation without relying on low- density guidance. One viable method is temperature pling (Ackley et al., 1985), method commonly used in tra- ditional likelihood- based frameworks (Ackley et al., 1985; Kingma & Dhariwal, 2018). the context of diffusion mod- els, temperature sampling can be implemented by scaling the score function along the generation trajectory (Dhariwal & Nichol, 2021). For instance, in the empirical reverse VP- SDE that employs pretrained score function s0 t), this translates to:
dæ = [ 1 3(1)))== ((t) logmt(a)) dt + di(t)dwr.
(6)
Simulating Eq. (6) with score model so(x,, t) (trained on the VP-SDE forward kernel) generates sample trajectories whose marginal distributions approximate {pr(w)) (Song et al., 2020); this is often called an em- pirical reverse VP-SDE.
Discrete VP-SDE (DDPM). Consider N discrete points lin- early distributed across timesteps E [o, TJ. ].. A discretized expression of the forward SDE in Eq. (3) is (Song et al., 2020):
dæ 8()x B() s0 t)
dt + v B(t)dw, (9)
where is the temperature parameter. Since sp 1)/T R V log (x) this sampler has been regarded as pro- ducing trajectories along { 1/TTT (Dhariwal & Nichol, 2021), where Z, is normalization constant. In this context, choosing > 1 (i.e., high-temperature) is expected to favor generation in low-density regions.
2; = V1 3,04-11 + V Biz2, 1= (7)
where 2, (). (3,),, denotes discretized se- quence of B(t). Unfolding Eq. (7) across discrete timesteps i yields one-shot perturbation: I; Va vI a,z,
However, we argue that this naive application of high- temperature sampling is ineffective within the diffusion
where A(0, I), and a; I1 aj for a; - Br.
3

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
model framework. In fact, the marginal densities of trajecto- ries generated by Eq. (9) are distinct from { 1 (t)))))/r) )t=o- potentially leading to unwanted generation results. A formal description is provided below, with proof in Appendix B.1.
sx()) ~ ~N I). we propose a 7-scaled initialization:
((T) ~ ~N 7 7>1,
(11)
where the range of y is selected to enhance minority sam- ple generation. Intuitively, this modification encourages initializations from low-density regions of the terminal dis- tribution pr, potentially facilitating the generation of under- represented samples in the data distribution Po-
Proposition 3.1. Consider the temperature-scaled reverse VP-SDE in Eg. (9). Assuming the score function is op- timal. i.e. se(x,t)
v log p((),, the marginal densi- ties of samples generated by this SDE are not equal to 1 (x()))))TT It= in general.
However, our first modification alone, which is reminiscent of the truncation trick (used in Brock et al. (2018)), provides limited improvement. We found that this is particularly pronounced in diffusion models with negligible terminal signal-to-noise ratio (SNR), i.e., (T)) s 0 for large T in Eq. (5). This is because, with a near-zero terminal SNR, the influence of the we--dessttyencourrgee (T) diminishes due to multiplication by negligible aT)), resulting in a marginal effect on the generative process. We will later provide theoretical justification of this claim.
One could argue that as long as high-quality minority in- stances are effectively generated, is not strictly necessary for trajectory samples to adhere to the expected marginal densities { Pe(z)) 1/7 (t))">/r" Jt=o- However, temperature sam- pling presents a critical practical limitation: it often suffers from significant errors in score function estimation along the sampling trajectories, potentially leading to degraded sample quality. This is direct consequence of the score function's 's scaled amplitude (1/T), which causes interme- diate samples a, to deviate from noisy data manifold M, on which the diffusion model is trained to denoise. These observations are consistent with the limited performance of temperature sampling reported in Dhariwal & Nichol (2021). See Figure 3 for our empirical analyses on this point.
initial point
Timestep-skipping. We address the vanishing impact prob- lem by skipping several of the carliest timesteps. More specifically, we start simulations of Eq. (10) from:
Taip == T -Sssppp
(12)
Another possible solution to guidance-free minority gener- ation is to employ the truncation trick (Brock et al., 2018: Karras et al., 2019) popularly employed in GANs (Good- fellow et al., 2014). This technique involves sampling the latent vector from a truncated normal distribution, which is often implemented via suriance-scaled Gaussian (Brock et al., 2018). However, as detailed in Section 3.3, naive application of this technique to diffusion models yields only marginal improvements in minority sample generation; see Table 2a for empirical results.
where Aski represents the amount of skipping. which is selected to ensure non-negligible a(Tasp)) We found that integrating the timestep skipping with variance boosting results in a significant synergistic improvement in minority sample generation.
While employing the timestep skipping alone could often yield some performance gains, we emphasize that these improvements are limited and inferior to the combined ap- proach; see Table 2b for details.
Validation on toy data. In Figure 2, we provide a sanity check of Boost-and-Skip
3.2. Boost-and-Skip: minority-focused generation with two simple tweaks
two-dimensional toy dataset comprised of two concentric circles. There, we verify that variance-boosting and timestep-skipping with reverse-SDE (Figure 2())) is the only combination which generates on- manifold minority samples ablating boosting, skipping, or SDE leads to inferior results. In the following section, we provide theoretical intuition to why all three components are necessary for successful minority generation.
In this section, we present Boost-and-Skipp a novel guidance- free approach that sidesteps the practical limitation dis- cussed in the previous section. A key benefit of Boost-and- Skip is its simplicity combined with significant improve- involves
ments and theoretical grounding. Boost-and-SS two core modifications to the standard stochastic reverse diffusion process: (i) boosting the variance of the initial noise; and (ii) skipping several carly timesteps.
3.3. Rationale behind Boost-and-Skip
Variance-boostingg. Consider an empirical reverse VP-SDE implemented using the score model so so(rr,)) t):
Now we delve into the underlying principles of Boost-and- Skip, elucidating why and how it is effective for minority generation. We present two key mechanisms: (i) low-density emphasis and (ii) rectification through contraction.
dx= = -()0)=((()s0000)))) di + v VB(f)d.. (10)
Low-density emphasis. We argue that Boost-and-Skin encourages sampling from low low-density instances by ampli- fying their probability densities. To see this, we consider a
In contrast to the common ignition practice that employs
4

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
simple (yet non-trivial) scenario where Po follows a multi- variate Gaussian distribution.
where we define 1 (x) 0 when K. >
Proposition 3.2. Let the data distribution be x(0) (N Eo), and assume the optimal score function so(x, t) = V log (a) trained on Po via the forward SDE in Eq. (3). Suppose the reverse SDE in Eq. (10) is initial- ized with @(Taip) ~ (i Eran). Then, the resulting generated distribution corresponds to (0) ~ (i Eo), where
In Appendix A.2, we provide an illustration that exhibits
the behavior of Σg og under the conditions specified
in Corollary 3.3; see Figure 6 therein.
Rectification via contraction. A potential concern is whether the amplified noise components due to variance- boosted initialization may impede high-quality genera- tion. To address this, we invoke stochastic contraction the- ory (Pham, 2008; Pham et al., 2009), principle that is often used to describe error-rectifying behaviors of stochas- tic diffusion generative processes (Chung et al., 2022; Xu et al., 2023). Specifically under the discrete VP-SDE setting in Equation (8), we establish general theoretical result (15) showing that, for an arbitrary distribution po. the error in- troduced by the boosted initialization decays exponentially as stochastic sampling progresses. See below for a formal description of the claim.
Pu = Mo + a(Taip)) En D0p (iTTe mmpp) (13)
Tup
En Eo + aa( (Tatipp) $10
(Er ETar)-
(14)
Tap
Here, aT and Eiap are defined as:
TT = (T(TTttppp)) M40 M40
ETap" I + (TTapp) )2 (Eo 1).
(16)
See Appendix B.2 for the proof. In the considered Gaus- sian setting, Boost-and- -oost-and-Skkp Skip can be instantiated by set- ting ΣTap = -²I I and saip < T. Note that when STao TTa, > 0, the initialization contributes to amplify the variance of the resulting generated distribution Σn com- pared to the original Σg (see Eq. (14)). This indicates that our approach can lead to probability increases in low-density regions of Po. i.e., the effect of low-density emphasis.
Proposition 3.4. Consider an empirical version of the dis- crete VP-SDE in Eq. (8):
1
ax, + +(1 aa))ss((( + v1 a;z, I;-1 Vo;
where i E {I....,N} N). Assume the optimal score func- tion, i.e., so(x, V~ Pt)), which is trained an arbitrary data distribution po- Consider two sample trajec- tories { and ( where Nssip N, which are initialized with distinct distributions: EN ~ N I) and NN ~ (0, 2 Assuming that {æ; J is bounded process such that |æxill2 < B (as in Xu et al. (2023)), the expected error between samples from these two trajectories Naai
A notable point here is that when Tsip se T, corresponding to the case where only boosting (i.e., our first modification) is applicd, the low-density emphasis impact does not mani- fest. This because limT (TTx))p) = (T) e 0 leads to the recovery of the original data distribution (Po: Eo) x (Mo, Eo); see Eqs. (13) and (14) for details. This highlights the necessity of incorporating time-skipping for effective minority generation, and also explains why the truncation trick (used in Brock et al. (2018)) could be insufficient in the diffusion model context. The key mechanism of Boost-and- Skip can be interpreted from signal processing viewpoint. See Appendix A.4 for detailed analyses on this perspective.
at step i E {0...., Naip - 1} is given by:
2C
E|| x. - 22 S
+ 2 - (B² + 2d)), (17)
1- X2
where X denotes the contraction rate:
1 11-0-11
To show that the condition for the low-density emphasis effect, i.e., ΣTa, ΣTa, > 0, can be satisfied in practice, we provide a corollary that characterizes the range of Tkpi over which the variance amplification impact occurs:
A=
max je(i+l, Napk
Va
(18)
1 1-aa āj
and C' == d(1 - -NNa))
Corollary 3.3. Suppose Eo = I and ΣTAY 2,,,, and define the quantity (if it exists)
See Appendix B.4 for the proof. Notice that the error term B² + 7 d decreases exponentially w.r.t. Nakip with contrac- tion rate X. In Appendix A.3, we provide more general contraction theory result that shows PF-ODE preserves (to- tal variation) distance between distribution, whereas reverse- SDE contracts distributional distances. In other words, PF. ODE propagates initial distribution error throughout the generation process. This implies that stochasticity is critical for initial error contraction, and also explains why a naive combination of Boost-and-Skii and ODE- samplers fail to produce high-quality minority samples.
K === V(> 10/000/01101100011 -
The variance-amplificatton effect of Eo occurs iff
Ø
i < 1 S oo:
(a- (k)) x)
i < 7, do,
Tatip E
10, -1 (Ix)) 00)
if No
if < 1 < 7 and (oo, 7) # 1,
5

---

Boost-and-Skip: A Simple Shiddnce--Feee Diffusion for Minority Generation
D0/M
DUMM
1S 1100
1025 by A0]
T5 Li)
BUS (y-
000
(r 1.35)
10-
w/jo SDS (y 10
RDPM
DOPM
12
005
15 (r-- 1.05)
415
BSS
65-
15 1.55
</6 5DE
1.0
160
Diffrcion timodee
Diftaion rimnerp
Diffrrinn timestep
Diftadon thnndee.
(a) Trajectories of estimation error
(b) Norm trajectories
Figure 3: Trajectory analysis across various samplers. "TS" refers to temperature sampling (Ackley et al., 1985), and "w/ SDE" indicates our approach without stochastic sampling, i.e., using PF-ODE. "B&S" is ours, Boost-and-Ste is the temperature parameter used to scale the score function (e.g., $0(;;; i)/T), and y controls the boosted initialization strength. (a) Noise estimation errors across discrete timestep i, where x := v aa + VI- a,e. Here Tøje represents denoised sample from zi, i.e., the posterior mean of x While temperature sampling fails even with slight variations in T. Boost-and -nokkpp performs as well as DDPM, demonstrating its robustness to imperfect score models. (b) Norm of intermediate samples x; across timestep i. We see that norm trajectories of our approach rapidly converge to that of DDPM, exhibiting the contraction effect claimed in Section 3.3. In contrast, trajectories of temperature sampling diverge from the DDPM curve, further revealing its pathology. See Figure 4 for visualizations of intermediate samples over trajectories.
4. Experiments
(a)
Datasets and pretrained models. Our experiments were conducted on four benchmarks settings with varying res-
(b)
olutions: (i) CelebA 64 x 64 (Liu et al., 2015); (ii) 256 x 256 (Yu et al., 2015); (ii) Im-
(c)
(d)
ageNet 64 x 64 (Deng et al., 2009); and (iv) ImageNet 256 x 256. We consider unconditional generation on CelebA and LSUN- Bedrromms, while performing dlass-condiiinall generation the ImageNet settings. The CelebA pretrained model was developed in nn-house, adhering to the configura- tion described in Um et al. (2023). The pretrained models for LSUN-Bedroome and ImageNet were taken from the checkpoints provided by Dhariwal & Nichol (2021).
Figure 4: Denoised intermediate samples along genera- tive trajectories CelebA (Liu et al., 2015). (a) DDPM.
(b) Temperature sampling with = 1.1. (c) Ours with ? 5.0. (d) Ours with y = 1.1 and PF-ODE. From left to right, discrete timestep i decreases from i = 100 to 0. Leveraging the contracting nature of stochastic genera- tion, Boost-and-Skip reduces excessive noise introduced by boosted initialization during early stages of generation.
Baselines. We evaluate our approach against a diverse array of baseline methods, including frameworks beyond diffu- sion models and minority samplers. Specifically, we in- clude two
GAN frameworks for compar- ison: BigGAN (Brock et al., 2018) and StyleGAN (Kar- ras et al., 2019). Additionally, we incorporate prominent diffusion- based approaches with their standard sampling techniques: ADM (Dhariwal & Nichol, 2021), LDM (Rom- bach et al., 2022). EDM (Karras al., 2022), and DiT (Pee- bles & Xie, 2022). The state-of-the-arr diversity sampler, CADS (Sadat et al., 2023), is also included for benchmark- ing. For minority sampling baselines, we consider the most advanced diffusion- based approaches: (i) Sehwag et al. (2022); (ii) Um et al. (2023); (iii) ADM-MI. (Um et al., 2023) (iv) Um & Ye (2024b); and (v) temperature sampling.
In practice, the score function (x(x),) t) is not optimal and may introduce estimation error. This is particularly relevant during early stages of our generative process, where the score model is tasked with estimating boosted-noise sam- ples that were not encountered during training. Nonetheless, our empirical findings indicate that within reasonable ranges of 7, the contraction effect inherent stochastic sampling rapidly corrects initial error during the early stages of genera- tion. We highlight that this is in stark contrast to temperature sampling, which is highly sensitive to the choice of 7 (i.e., the scaling parameter) and prone to collapse even with small deviations from r = 1. Figures 3 and 4 provide empirical analyses that investigate the robustness of our approach to imperfect score models; see details therein.
Evaluation Metrics. We employ a set of quality and diver- sity metrics to evaluate generated samples. Specifically, we use: (i) Clean Fréchet Inception Distance (cFID) (Parmar et al., 2022); (ii) Spatial FID (sFID) (Nash et al., 2021);
6

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
(a) ADM (Dhariwal & Nichol, 2021)
(b) Temperature sampling
(c) Boost-and-Skip (ours)
Figure 5: Sample comparison on ImageNet 256 x 256. Generated samples from two classes are exhibited: "jack-o'-lantern" -lantern"
(top row) and "bald eagle" (bottom row). We share the same random seed across all three approaches.
and (iii) Improved Precision & Recall (Kynkäänniemi et al., 2019). To evaluate the proximity to real minority data, we follow the approach used in previous works Um al. (2023); Um & Ye (2024b). Specifically, the lowest likelihoods (i.c., those with the highest AvgkNN values) as reference real data for calculation of quality and diversity metrics. Additionally, to compare the capability to generate low-density instances, we adopt three uniqueness measures: (i) Average k-Nearest Neighbor (AvgkNN); (ii) Local Outlier Factor (LOF) (Breunig et al., 2000); and (iii) Rarity Score (Han al., 2022). For all these measures, higher value indicate that instance is less similar to its neighborhood (Sehwag et al., 2022; Han al., 2022).
par with computationally intensive guided samplers across all datasets. We emphasize that these substantial improve- ments are achievable with minimal computational overhead; see Table 3 for a complexity analysis on ImageNet 64 x 64. In addition to its competitive performance in quality and diversity, our approach also performs well in neighborhood density metrics; see Appendix D.1 for explicit details.
employ instances with
Ablation studies. Table 2 presents ablation results on our design choices, y and A. We see that applying either modification individually results limited improvements (see Tables 2a and 2b). In contrast, combining both tech- niques with properly chosen 7 values yields significant en- hancements in minority generation; see Table 2c for details. In Appendix A.5, we present a further ablation exploring performance with y < 1.0, where we demonstrate a quality- improving potential ours; see Table for details.
4.1. Results
Qualitative comparisons. Figure 5 presents generated sam- ples from three different approaches on ImageNet 256 256. We observe that our framework consistently produces sam- ples with highly more distinct and intricate visual aspects compared to the baselines, characteristics often associated with low-density instances (Serrà et al., 2019; Arvinte et al., 2023). We highlight that this contrasts with temperature sampling yielding marginal changes from the baseline ADM, as further confirmed by our quantitative evaluations (see Ta- ble I). Additional samples including those from other bench- marks and basclines are provided in Appendix D.2.
Downstream application. To further highlight the prac- tical importance of Boost-and-Skip. we investigate its po- tential application in classifier training with synthetically augmented datasets. Specifically, we examine whether our minority--romoote generated samples can enhance classi- fication performance. Following Um & Ye (2024b), we consider the prediction of 40 attributes in CelebA and train ResNet-18 models on four datasets: (i) the CelebA train- ing set; (ii) CelebA augmented with 50K samples from ADM (Dhariwal & Nichol, 2021); (iii) CelebA augmented with 50K samples from Um & Ye (2024b); and (iv) CelebA augmented with 50K samples from ours. As in Um & Ye (2024b), we use an off-the-shelf classifier to label the gener-
Quantitative evaluation. Table compares performance in terms of quality and diversity. Observe that our sampler achieves high-quality minority generating performance on

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Method
cFID
sFID 4
Pree
Rec
Method
cFID
sFID 4
Prec f
Rec
CelebA 64x64
LSUN Bedrooms 256 256
ADM (Dhariwal Nichol, 2021)
75.41
17.11
0.97
0.23
ADM (Dhariwal Nichol, 2021)
63.30
8.00
0.89
0.15
BigGIAN (Brock et al,, 2018)
80.58
16,80
0.97
0.19
L.DM (Rombach et al., 20122)
63.53
7.73
0.90
0.13
StyleGAN (Karras et al., 2019)
57.17
7.78
0.89
0.14
ADM-ML (Um et al., 2023)
51.99
13.40
0.94
0.30
Schwag et (2022)
28.25
10.64
0.82
0.42
Um (2023)
41.75
7.26
0.87
0.10
Um et al. (2023)
27.32
8.66
0.89
0.33
Um & Ye (2024h)
36.94
5.13
0.87
0.15
Um & Ye (2024bb
19.34
8.85
0.82
0.47
Temperature sampling
51.81
8.22
0.82
0.21
Temperature sampling Boost- Skip (proposed)
37.78
14.49
0.79
0.38
Boost- and-Skip Skip (proposed)
44.77
6.70
0.78
0.20
19.79
0.77
ImageNet 256 x 256
ImageNet 64x 64
ADM (Dhariwal & Nichol, 2021)
13.22
7.66
0.86
0.39
ADM (Dhuriwal & Nichol, 2021)
18.37
5.39
0.79
0.53
DiT (Peebles & Xie, 2022)
21.51
6.76
0.80
0.46
EDM (Karras al.. 2022)
19.09
4.73
0.73
0.59
CADS (Sadat al., 2023)
15.95
6.18
0.81
0.48
Schwag et al (2022)
11.37
4.69
0.80
0.52
Sehwag et al. (2022)
10.93
6.66
0.85
0.39
Um et (2023)
12.47
3.13
0.76
0.56
Um (2023)
11.44
4.63
0.85
0.42
Um & Ye (2024b)
11.24
3.17
0.73
0.62
Um & Ye (2024b)
9.98
4.35
0.83
0.45
Temperature sampling Boost-and-Skip (proposed)
16.19
4,20
0.76
0.58
Temperature sampling
12.48
6.72
0.84
0.41
12.68
3.18
0.74
0.61
Boost-and (proposed)
10.04
4.33
0.83
0:45
Table I: Quantitative comparisons. "ADM-ML" represents a classifier-guide -uiieerr-guded baseline that implements conditional
generation on Minority Labels (Um et al., 2023). For baseline real data to compute the metries, we employ the most unique samples that yield the highest AvgkNN values, following the previous convention (Um et al., 2023; Um & Ye, 2024b). The best results are marked in bold, and the second bests are underlined.
72
cFID
sFID
Prec f
Rec
A.
cFID
sFID
Pree
Rec
72
cFID 4
sFID
Pree
Roc 1
1.0
84,98
23.07
0.98
0.14
84,98
23.07
0.98
0.14
1.0
74.41
20.50
0.97
0.21
4.0
81.75
22.71
0.97
0.17
61.86
19.97
0.96
0.13
2.0
51.47
16.43
0.94
0.33
9.0
82.10
22.78
0.98
0.15
20
55.15
22.02
0.91
0.06
4.0
23.56
12.17
0.77
0.50
16.0
83,32
22.91
0.98
0.18
50
289.70
119.59
0.29
0.00
9.0
219.21
45.59
0.05
0.67
(a) Boost-only (i.e, A = 0)
(b) Skip-only (ie.. 7 1.0)
(c) Boost-and- Skip (with A = 3)
Table Exploring the design space of Boost-and-Skip. "Boost-only" refers to using boosted initialization alone, while "Skip-only" represents configurations that only employ the proposed timestep skipping. 7 denotes the boosting scale, and A. indicates the number of timesteps skipped. Using either technique individually results in limited performance improvements.
Complexity
Training data
Acc
F1 f
Prec
Rec
Method
cFID L
sFTD
Infer
Extra
Memory
CelebA trainset
0.898
0.746
0.815
0.710
ADM M 2011)
18.37
MB
ADM gens (50K)
0.897
0.742
0.808
0.711
HDM
19.09
MB
Schwag (2022)
>
187.04 MB
+ SGMS gens (50K)
0.903
0.757
0.822
0.724
et (2023)
>
+
Ours gens (50K)
0.902
0.755
0.819
0.723
Um (2024b)
11.24
2,80
386.75 MB
Tomperature sampling
Booos--and--e Skip (ours)
12.68
16.19
3.18
4.30
0.99
0.98 s
85.73 MB
85.73 MB
Table 4: Data augmentation for downstream classifica-
tion tasks. "SGMS" refers to the approach by Um & Ye (2024b). All settings were evaluated on the CelebA testset and averaged over three different runs.
Table 3: Comparison of complexity across existing sam-
plers on ImageNet 64 x 64. "Infer" represents the infer- ence time measured in seconds/sample. "Extra" refers to the additional time, in days, required to construct external classifiers used for guided sampling (Sehwag et al., 2022; Um et al., 2023). "Memory" indicates the peak memory us- age, measured in MB/sample. All measurements are based on a single NVIDIA A100 GPU.
incorporating variance-boostee initialization and timestep skipping, our framework promotes the emergence of un- derrepresented features without relying on additional low- density guidance. Theoretical and empirical results vali- dated its effectiveness, demonstrating competitive perfor- mance while significantly reducing computational costs. We further demonstrated its practical utility in downstream tasks such as data augmentation, highlighting its broader applicability in generative modeling.
ated samples. Our results show classification improvements comparable to Um & Ye (2024b), despite our method being significantly more computationally efficient, further demon- strating its utility in downstream tasks.
A limitation is that as discussed in Section 3.3, naive application to deterministic samplers may fail due to the absence of a contracting effect. A promising future direction is to address this limitation, extending the effectiveness of our approach to wider range of generative processes.
5. Conclusion
We introduced Boost-and-Skipp a simple yet impactful guidance-free approach for minority sample generation. By
8

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Impact Statement
In 2009 IEEE conference on computer vision and patlern recognition, pp. 248-255. leee, 2009.
One potential negative impact is the intentional misuse of our sampler suppress the generation of minority-feature samples. This could be achieved by setting V < 1.0 with A O, potentially biasing the initialization toward high- density regions (as explored in Appendix A.5). Recognizing and mitigating this risk is essential, emphasizing the im- portance of responsible deployment to ensure inclusivity in generative modeling.
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Du, X., Wang. Z., Cai, M., and Li, Y. Vos: Learning what you don't know by virtual outlier synthesis. arXiv preprint 192. 2022.
Du, X., Sun, Y., Zhu, X., and Li, Y. Dream the impossible: Outlier imagination with diffusion models. In Advances in Neural Information Processing Systems, 2023.
Acknowledgments
This work was supported by the National Research Foun- dation of Korea under Grant R8S20244003336454 and the Institute of Information & Communications Technology Planning & Evaluation (ITP) grant funded by the Korea government (MSIT) TRS-2025-02304967, AI Star Fellow- ship (KAIST)).
Goodfellow, I., Pouget-Abadie, J., Mirza, M.. Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances in neural informa- tion processing systems, 27, 2014.
Han, J., Choi, H., Choi, Y., Kim, J., Ha, J.-W., and Choi, J. Rarity score: A new metric to evaluate the uncommonness of synthesized images. arXiv preprint arXiv:2206.08549, 2022.
References
Ackley, D. H., Hinton, G. E., and Sejnowski, T. J. A learning algorithm for boltzmann machines. Cognitive science, 9 (1):147-169, 1985.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Arvinte, M., Cornelius, C., Martin, J., and Himayat, N. Investigating the adversarial robustness of density esti- mation using the probability flow ode, arXiv preprint vrXiv:2310.070000 2023.
Ho, and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. Lof: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93- 93-104, 2000.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Brock, A., Donahue, J., and Simonyan, K. Large scale gan training for high fidelity natural image synthesis. arXiv preprint ar / /096, 2018
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, D. Video diffusion models. Advances Neural Information Processing Systems, 35:8633-8646. 2022.
Choi, K., Grover, A., Singh, T., Shu, R., and Ermon, S. Fair generative modeling via weak supervision. In Proceed- ings of the 37th International Conference on Machine Learning (ICML). PMLR, 2020.
Hu, M., Zheng, J., Zheng, C., Wang, C., Tao, D., and Cham, T.-J. One more step: A versatile alug-and-play module for rectifying diffusion schedule flaws and enhancing low low-frequency controls. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7331-7340, 2024.
Chung. H. Sim, B., and Ye, J. C. Come-closer-diffuse faster: Accelerating conditional diffusion models for in- verse problems through stochastic contraction. In Pro- ceedings of the IEEF/CVE Conference Computer Vi- sion and Pattern Recognition, pp. 12413-12422, 2022.
Huang, G. and Jafari, A. H. Enhanced balancing gan: Minority- image generation. Neural computing and applications, 35(7):5145-51) 51554,,,,,,,,, 2023.
Corso, G., Xu, Y., De Bortoli, V., Barzilay, R., and Jaakkola, T. Particle guidance: non-iid diverse sampling with diffu- sion models. arXiv preprini arXiv:2310. 13 13102, 2023.
Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401 4410, 2019.
Deng, J., Dong, W., Socher, R., Li, L.JJ., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database.

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35: 16565-26577 2022.
L., al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. arXiv preprint arXiv:2212.09745 2022.
Kingma, and Dhariwal, P. Glow: Generative flow with invertible Ixl convolutions. Advances in neural information processing systems, 31, 2018.
Pham, Q-CC. Analysis of discrete and hybrid stochastic
systems by nonlinear contraction theory. In 2008 10th In- ternational Conference on Control, Automation, Robotics and Vision, pp. 1054-1059 IEEE, 2008.
Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., and Aila, T. Improved precision and recall metric for assess- ing generative models. Advances Neural Information Processing Systems, 32, 2019.
Pham, Q.-C., Tabareau, N., and Slotine, J.JJ. A contraction
theory approach to stochastic incremental stability. IEEE Transactions on Automatic Control, 54(4):816 2009.
Lin, S., Liu, B., J., and Yang, X. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 3404-5411, 2024.
Plancherel, M. and Leffler, M. Contribution à l'étude de la
représentation d' l'ue fonction arbitraire par des intégrales définics. Rendiconti del Circolo Matematico Palermo (1884-1940), 30(1 (129)))2993333) -355, 1910.
Lin, Z., Liang, H., Fanti, G., Sekar, V., Sharma, R. A., Soltanaghaci, E., Rowe, A., Namkung, H., Liu, Z., Kim, D., et al. Raregan: Generating samples for rare classes. arXiv preprint arXiv:2203. 2022.
Qin, Y., Zheng, H., Yao, J., Zhou, M., and Zhang, Y. Class- Blass-balancing diffusion models. arXiv preprint 0tti:2305.00562.. 2023.
Liu, Z.. Luo, P., Wang. X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
) Lu, J., Teehan, R., and Ren, M. Procreate, don' 't reproduce! propulsive energy diffusion for creative generation. arXiv preprint ar arXiv:2408.022 02226, 2024.
Naeem, M. F., Oh, S. J., Uh Y., Choi, Y., and Yoo, Reli- able fidelity and diversity metrics for generative models. In International Conference on Machine Learning, pp. 7176-7185. PMLR, 2020.
Sadat, S., Buhmann, J., Bradely, D., Hilliges, O., and Weber, R. M. Cads: Unleashing the diversity of diffusion models through condition-annealee sampling. arXiv preprint arXiv:2310.17347 2023.
Nash. C., Menick, J., Dieleman, S., and Battaglia, P. W. Generating images with sparse representations. arXiv preprint arXiv:2103.03844 2021.
Samuel, D., Ben-Ari, R., Raviv, S., Darshan, N., and Chechik, G. It is all about where you start: Text-to- image generation with seed selection. arXiv preprint arXhv:2304.14538 2023.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2/ 2021.
Sehwag, V., Hazirbas, C., Gordo, A., Ozgenel, F., and Can- ton, C. Generating high fidelity data from low-density regions using diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11492-11501 2022.
Nichol, A. Q. and Dhariwal,
P. Improved denoising diffusion probabilistic models. In International Conference Machine Learning, pp. 8162-8171. PMLR, 2021.
Serrà, J., Álvarez, D., Gómez, V., Slizovskaia, O., Núñez, J. F., and Luque, J. Input complexity and out-of- distribution detection with likelihood- generative models. arXiv preprint arXiv: 1909. 1480, 2019.
Parmar, G., Zhang, R., and Zhu, J.-Y. On aliased resizing and surprising subtleties in gan evaluation. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 11410-11420, 2022.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint 1.Ni:20/1.13444 2020.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga,
10

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Um, S. and Ye, C. Minorityprompt: Text minority image generation via prompt optimization. arXiv preprint arXiv:2410.07838, 2024a.
Um, S. and Ye, J. C. Self-guided generation of minor- ity samples using diffusion models. arXiv preprint vrxiv:2407.11555, 2024b.
Um, S., Lee, S., and Ye, C. Don't play favorites: Mi- nority guidance for diffusion models. arXiv preprint arXiv:2301.1233,, 2023.
Vincent, P. A connection between score matching and de- noising autoencoders. Neural computation, 23(7):1661- 1674, 2011.
Xu, Y., Deng, M., Cheng, X., Tian, Y., Liu, Z., and Jaakkola, T. Restart sampling for improving generative processes. Advances in Neural Information Processing Systems, 36: 76806-76838, 2023.
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., and Xiao, J. Lsun: Construction of a large-scale image dataset using deep learning with humans the loop. arXiv preprint arXiv: 03360, 2015.
Yu, N., Li, K., Zhou, P., Malik, J., Davis, L.,, and Fritz, M. Inclusive gan: Improving data and minority coverage in generative models. European Conference on Computer Vision, pp. 377-393. Springer, 2020.
Zhang, C., Zhang, C., Zheng, S., Zhang, M., Qamar, M., Bae, S.-H., and Kweon, S. A survey on audio diffusion models: Text to speech synthesis and enhancement in generative ai. arXiv preprint 2iti::2333.133336 2023.
Zhao, Y., Nasrullah, Z., and Li, Z. Pyod: A python toolbox for scalable outlier detection. Journal of Ma- chine Learning Research, 20(96):1-7, 2019, URL http: 111. hhmll.
// jmmr.. www.or//poo..... hpppps///200///

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
A. Additional Discussions, Ablations, and Analyses
A.1. Related work
In addition to closely related studies mentioned Section I (Se- hwag et al., 2022; Um et al., 2023; Um Ye, 2024b;a), several other works explore distinct conditions and scenarios in the con- text of minority generation (Yu et al., 2020; Lin al., 2022; Qin et al., 2023; Huang & Jafari, 2023; Samuel et al., 2023). One instance is Qin et al. (2023) wherein the authors develop training technique to mitigate a class imbalance issue when constructing class-conditional diffusion models. A distinction w.r.t. ours that they require a specialized training, and their method is limited to conditional diffusion models. Another notable study is done by Samuel et al. (2023). Specifically using text-to-image (T2I) diffusion models (Rombach et al., 2022), they develop a sampler to faithfully generate samples of unique text-prompts by employing real reference data instances associated with the given prompts. The key distinction to ours is that their method is limited to T2I models and rely upon a set of real reference data.
1 1.0
12
7
7 2.0
10
7 2.5
== 3.0
oB (Ref)
0.0
0.2
0.5
skip
T
Figure 6: Low-density emphasis impact of Boost-and- Skip. We visualize og (i.e., the scale of En == og I) across Tsip/ /T under the settings specified in Corol- lary 3.3, with surpasses that of Eo for y > 1 and Takp < T. demon- strating the low-density encouraging influence of the Boost-and-Skii approach.
related yet distinct line of research is to improve the diversity of conventional diffusion samplers (Sadat et al., 2023; Corso et al., 2023; Lu et al., 2024). For instance, Sadat et al. (2023) propose simple conditioning technique to boost-up the diversity by in- troducing time-schedueed noise perturbations in the conditional embedding space. The difference from our study is that their method is confined to conditional diffusion models and not specifically designed for minority generation. The approaches in Corso al. (2023); Lu al. (2024) share similar spirits, repelling intermediate latent samples within an inference batch to produce visually distinct outputs. However, as with Um Ye (2024b:a), these methods often introduce computational challenges, e.g., due the reliance on backpropagation.
2. Observe that the variance of En
A.2. Illustration of the impact of low-density emphasis
We continue from Section 3.3 and provide visualization on the effect of low-density emphasis Boost-and-Siee aasst.pnd-Skip.. Let us first recall Corollary 3.3:
Corollary 3.3. Suppose Eo = of and ΣT.. = 21, and define the quantity (if it exists)
K v (72 1) -
The suriaece-ammpiffcatiin effect of En occurs iff
Ø
if S 1 so.
(a (n)) (0)
if1 < 7000,
Tokip E
10, (x))
(x))
if > 7,00: if $ 1 S y and (oo, 7) # 1,
00)
where we define a (x) := 0 when > 1.
Figure illustrates the behavior of Σn = 0 under the conditions specified in Corollary 3.3, where To = 2. We see that the variance scale of En exceeds that of En for > 1 and Taip < T. demonstrating the variance-boosting effect induced by the proposed modifications. Notably, regardless of the value y, this amplification effect does not occur for Tikip Rs T, thus supporting the effectiveness of our time-skippen skipping technique for promoting low-density emphasis.
A.3. Further results on contraction theory
We continue from Section 3.3 and extend our analysis of contraction theory in the context of foost-and-Skipp To this end, we first explore its behavior in non con-stochastic generative processes that lack the contraction effect. Specifically, we consider
12

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
its application the probability flow ODE (PF-ODE) (Song et al., 2020) associated with Eq. (10):
1 (() dæ [
1 _ (()))s((z,,)) dt.
(19)
2
The following proposition characterizes the density of generated samples PODE when going through the above ODE under the same settings Proposition 3.2:
Proposition A.1. Consider the same data distribution po and the optimal score function sa(x, 1) as Proposition 3.2. Suppose the PF-ODE in Eq. (19) is initialized with 20 (TTaip) ~ N (feras Eyary ). Then, the resulting distribution t 0 is given by (0) ~ ~N Eo,ODE), where
Po,DDE = 140 + 2/ Iap (p mT
Σ 1/2
(20)
Eo,ODE Eg 1 Eray.
(21)
Here, ae and Ei are defined as in Eqs. (15) and (16), respectively.
We leave the proof in Appendix B.5, We in Eq. (21) that the variance Eo is multiplied by the initial variance TTa.. This implies that the low-density emphasis impact observed in the SDE-case may also manifest in the considered ODE case. However, the mean and variance formulas of the ODE-generated distribution PODE lack the ) contributes to the recovery of Po. indicating the inability of (non-stochastic) ODEs for error rectification. Below, we present a weaker but more general stochastic contraction result which shows that similar phenomenon occurs with VP-SDE with general data distributions.
that
Proposition A.2 (Error contraction for VP-SDE diffusion models). Assume the same setup Theorem 3 in Xu et al. (2023),
i.., ||tV log (x) < L1 and || < for any Dt in the support of Pt and reverse-SDE trajectories. Let
SDD x Erii
SDE@E (rue mmax + bmin), ) y Vtmm SDE(y bmaxs Xmax 7 kmmin) = ODE ODE(x tmax + fmin), ODE
(22)
ODE 2 fani
ODE (3/ Imax" fmax - tmiu)
(
(23)
where SDE and ODE denote solutions to the reverse-SDE and PF-ODE, respectively. Then
(2
ODE
y ODEE)
TV TVV(xtma tmax Yr Ummax
(24)
B S (1 1--QQ /a((tmax) 2 =aa(fii
TV(xSDE
Emit y SDE)
c Bla BLa /tmin []a(tmax) - /2//12 /fain
TV TVttmmas Y ylmar)
(25)
where Q(r)= = I( 2 for 1).
The proof can be found in Appendix B.6. Here, TV denotes the total variation distance. Indeed, we observe that PF-ODE does not contract any initial error. On the other hand, the multiplicative factor on the RHS of Eq. (25) shows that reverse- reduces initial error. We empirically found that this absence of the rectifying capability often makes ODE-based samplers struggle to generate high-quality minority samples with Boost-and-Skip See Figures 2, and for instance.
A.4. How Boost-and-Skip works: a signal processing perspective
We continue from Section 3.3 and investigate the principle of Boost-and-Skip in signal processing viewpoint. As explored in Lin et al. (2024); Hu et al. (2024), generated samples from diffusion models are often affected by initial Gaussian noise, particularly in shaping oow-frequeccy components of images (like brightness). This effect is especially pronounced in diffusion frameworks that employ noise schedules with non-zero terminal SNR (i.e., (<T) 36 0), allowing low-frequency components of the initial noise propagate into the generative process.
From this standpoint, Boost-and-Skip can be viewed as two-step approach. First, the low-frequency components of the noise are amplified through variance boosting (by Plancherel theorem (Plancherel & Leffler, 1910)), enhancing the visual variability of the random Gaussian noise. Second, this amplified low-frequency information is made more influential by initiating the generative process at a skipped initial timestep Trip with a non-zero terminal SNR (a(Tapp) & 0). We conducted an empirical analysis to support this perspective. See Table 5 for explicit details.
A.5. Extended ablation: effect of using 7 < 1.0
13

---

Boost-and-Skip: A Simple Guidance- Diffusion for Minority Generation
Semtott. LPF
cFID
sFID 1
Prec 1
Rec
Seunoff, HPE
cFID 4
sFID
Prec
Rec
64 (uncut)
23.56
12.17
0.77
0.50
(uncut)
23.56
12.17
0.77
0.50
48
23.38
12.31
0.77
0.50
8
60.16
18.26
0.95
0.23
32
23.82
12.32
0.77
0.51
75.68
21.33
0.97
0.16
16
28.46
14.34
0.84
0.49
32
95.50
25.88
0.98
0.10
33.08
14.98
0.84
0.44
100.03
28.26
0.97
0.12
(a) Low-pass filtered after boosting
(b) High-pass filtered after boosting
Table 5: Amplification of low-frequeney components is crucial to the success of Boost-and-Skip. The performance values were evaluated on CclebA, with low-pass or high-pass filters applied after the boosted initialization. feutott, LPF represents the cut-off frequency for the low-pass filter, while feuwit, HPF denotes the cut-off frequency for the high-pass filter. While the low-pass filtered cases maintain performance gains robustly across varying cut-off frequencies, high-pass filtering leads to immediate degradation, as seen in significant performance drop starting at fcutoff. HPF = 8. This suggests that the enhanced minority generation is primarily driven by the amplification of now-frequency components.
One may wonder: what if we employ Y 1.0, effectively reversing the direction of minority generation? We found that this oppositional choice often exerts interesting quality-enhancing effect on the generative process. Intuitively, initializing with y < 1.0 can be interpreted as starting generation from high-density region, which may guide the sampling process toward higher-quality outputs. Our empirical findings confirm this intuition.
7²
Prec
Rec t
Den
Cov t
1.0
0,851
0.627
1.290
0.940
0.8
0.874
0.588
1.384
0.938
0.6
0.881
0.586
1.495
0.936
0.5
0.879
0.565
1.478
0.926
Table Quality-enhancing effect of Boost-and- Skip on CelebA. "Den" and "Cov" denote Den- sity and Coverage (Naeem al., 2020), an addi- tional set of quality-diversity metrics. For com- puting these metrics, we used the CelebA test set as the baseline real data (rather than employing minority data as in Table 1) to assess the quality- diversity tradeoff within the ground-truth data manifold of CelebA
Table 6 presents the effect y on various quality and diversity metries for CelebA. Note that these metrics were computed using the test dataset, rather than minority data as was employed for Table 1. This allows us to examine the quality-diversity tradeoff from broader perspective, specif- ically within the tround-truth data manifold of CelebA. As shown in Table 6, using y < 1.0 improves sample quality at the expense of diver- sity, demonstrating that Boost-and-Ske serves as a control mechanism not only for diversity but also for quality.
Another key observation is that our approach is complementary to existing quality-enhaneie techniques, such as Classifier- Free Guidance (CFG) (Ho & Salimans, 2022). This suggests that Boost-and-Skip can be integrated with such methods to further refine generative performance, offering additional flexibility in balancing quality and diversity.
A.6. Discussion: Implications of Boost-and-Skip
We note that our framework has several important implications. The first one is that it improves the practical relevance of minority samplers by significantly reducing the computational overhead associated with existing methods. Secondly, our approach provides a simple mechanism for improving the diversity of diffusion models, a feature that has been largely absent in the research community. Another important point is to demonstrate that the pathological non-zero terminal SNR, arising from the training-inferent mismatch (Linn et al., 2024; Hu et al., 2024), can actually be advantageous. Lastly, we highlight potential opportunities in the initialization of stochastic sampling, a largely under-explored area compared to deterministic samples of diffusion models.
B. Proofs
B.1. Proof of Proposition 3.1
Proposition 3.1. Consider the temperature-scaled reverse VP-SDE in Eq. (9). Assuming the score function is optimal, i.e.,
se(x, t) = V log Pe (x)), the marginal densities of samples generated by this SDE are not equal to { (2) general.
1/117 //T]= in
14

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Proof. We first rewrite Eq. (9) using the ground-truth score function Ve Po under the optimality assumption:
dæ - - 3(t log dt + B(tjdw.
(26)
Let us assume that the SDE in Eq. (26) produces samples along ( (t) JEDD Then, we should have:
1 p((x) = pos (1) ) m Z
1 1/7 dæo,
(27)
where Z represents a normalization constant w.r.t We disprove this by providing a counterexample. Consider a dirac-delta density po and a VP-SDE forward kernel pus:
Po(oo) == 6(mr 14),
(28)
| V (a(t) 11
(or(x) (00) = 2
exp
(29)
2 - ax())
2(1 a(t))
where at)) represents the noise schedule, which is function of (t)). With this instantiation, we expand pa (x) on the LHS of Eq. (27) by using the definition of pe:
/ (a dry
1/7
/r(x)
(30)
1/7
/
d
1
|| (a)
exp
8(x0 1e) dæy
(31)
21T ca(1))
2(1 a(t)
d/r
1
| Va(t))11133 2r a())
exp
(32)
V 2 - at)))
Since (x) is Gaussian, we can determine the normalization constant Ze, yielding the following expression for the LHS of Eq. (27):
d
| - va(t))l1 2r(1 aa)))
Pc 1/) Ze
exp
(33)
21TT(1
On the other hand, the RHS of Eq. (27) is:
f ppc(aa (o(2)22)) 2 (T dæy- = / (V
1
1
d
| aa(t)2o113
2
-(())
exp
2(1 aa)))
z 8(20 -1)1/r dxp
(34)
| Va(t)ul3 2(1 - a(l))
exp
(35)
22 -a(())
which is not equal to the LHS (i.., Eq. (32)), egg., r # contradicting our initial assumption. This completes the proof.
B.2. Proof of Proposition 3.2 Lemma B.1 (Solution the VP-SDE). Let vp N Eo), and suppose æ; E (0, oo) evolves according to dx
1 2 3(1)rd + v B(1)dw
(36)
15

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
where 3(1) is a positive function which integrates xc. The forward process is Gaussian with mean and covariance
14 = a(l)
(37)
Σ; = I + (En I),,
(38)
where a(t) = e S6 P(s) such that lime-voc att) O.
Proof. Mean Ht of is governed by the ODE
dp./dl = EI (31))(( = -2,(1)
(39)
so its solution is given
H. 1, aa))000
(40)
The covariance E, of is governed by the ODE
/ dt
| - (x) - 11 B[ p,)=! E(
(41)
E| (z)); - (2) + (1)) - M2) - (4)) + +(t))
(42)
+ (m) - (0))))))((0)) 1 3(1)
(43)
(44)
-E; -I
(45)
whose solution is given (solve the ODE after making the change of variables Σ, Eg 1
E, = I I aat) - I)..
(46)
Since the initial distribution at t 0 is Gaussian and VP-SDE is a linear SDE, its solution is also a Gaussian.
Proposition 3.2. Let the data distribution be x0) NN Eo). and assume the optimal score function sæ t) V log Pc(x) trained on Po via the forward SDE in Eq. (3). Suppose the reverse SDE in Eq. (10) is initialized with a e(Taip) ~ N ErTgy. ) Then, the resulting generated distribution corresponds to x(0) N En), where
Go = 14 + ((Tship))
E (PrTem pTap))
(47)
En Eg a(Tssip)
)
22 Em (Draso ETar)
(48)
Here, Mr and Σx are defined as:
MTa = a aTTsip
(49)
ETar (Tshiip) ) (2E 1).
(50)
Proof. By Lemma B.1, the score function is given as
V (xg = Va Noglle, Ex) = -x - 1l)
(51)
such that the reverse-SDE is
dž, 1-1 - V(t) p( dt + /3(t) dw
(52)
(t())t + (())) (2. [1)] dt + /3tt) dw.
(53)
(()) I) dt - B(t) p4 dt + / 3()) dw:
(54)
where w; is reverse-time time Brownian motion. This is also linear SDE, and since TTan is assumed to be Gaussian. æo is also Gaussian. Hence, it suffices to derive the mean and covariance of &n. Fix some T > O and make the change of variables (9,, 8) & - T - 1) such that solving the SDE
dÿ. = - s)(227_ l)., ds + 3(T s)ET- -MT- ds VB(T 5) dw.
(55)
16

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
from to T is equívalent solving the original reverse-SDE Eq. (54) from / = T to O. We shall now derive the mean and covariance of y, 8 = T. To this end, define the eigen-decomposition
Σe QAQ", A = diag(An)
(56)
and recall that f4, = (4 and Σ, = =I (Et) 1). The mean ir of Ú, is governed by the ODE
dfp,/ds = EE| [ - (2E7T - 1)9, + B(T - 8) -T HI
(57)
= (T - -2 - - + 3(T
8) -- +
(58)
--ja(TT - -)) - Q diag( o(T - nl - (2
)Q T p4:
(59)
I+alf-- (A
+ aa( - 8) $)Q diag( Ita(T -8) )Q Mo.
(60)
Define o(z;a) == log(az² 1) log(z) such that
s))da s):d,, 1] dTs s(
(61)
ds
da(T-8)
d) de
a(T-s) -1a-)) -o -1( (- - s)a(T s)) (-1)
(62)
1+o(T
(aa(-aa)a0 -a)² -a)=(nn
= -
S)a(T ---
al -s) -a(T - (1n
(63)
1+a(T [ -1)
This motivates the transition matrix
V (s, sn) Q diag ( (la(T (aa() so):dn ):dn -1) Q T
(64)
diag( (A, (Tx(TT
aTT - xp)
(65)
alT
11-aa)a(7 -0 10
a(T mg) atT -8) Σx
(66)
with which we can calculate the solution to Eq. (60) as
j.. = (s, + So (1 r (T - o) - o) =QQ diag( 1 1) 11) up do
(67)
a)--s) Σx ET Po + PT
a(7)
a(T) a(T-s) ΣT- DT MT
(68)
a a(T-s} - Ex (in- UT)
(69)
where the integral in the first line is calculated as
So D (3) c)Q a) diag( ita(T -)-(,..11) Q T
Mo do ()n-1)a(T-))++1
(70)
1 Q diag( (Am (T) -)
00 a ( - 0) Q diag
(alag(((()))) a(T-o)
diag( ItalT- -)) Q Mo do
1
(71)
(T
(A, (T==))
diag( a(T-o)+3(T-)) ) Mo do
(72)
(L+a(T-o)((An11)))
Q diag(
(Aw 1))
diag(2 (2 5 2 [ - (T o) a(T-o)
(0)((-o)((mn1)))) d Mo
(73)
a(T o))
= Q diag(
(T((T diag 2 ss)
1QTHU uu
(74)
2((An - I)a(T 0.
Q diag
() 1))(T=))
a(T
diag(
(An [)((+=))+]]
(Am -1)(yaa7777] 0 T
x(T)
140
(75)
Q diag(
(a('-s)) -s)
a(T)²
)Q" T Ho
(76)
a(T (An 1)a[((]])*++
= aTT - 3) 5)H40
-)) Q diag(
aIT)
(An, 1)a (T
(A. - -1)a(1) 'Q (a(T))))
(77)
a(T) Σn ΣJ PT. PT as)
(78)
Setting s- T Taki in Eq. (69), we see that
E [bol = E(urayl
= Mras - Tas + a(Tuu - Pais)
o(Tuup
Ereo - Tas ET-
Twer (ElVol - MTa))
(79)
= Mo + ae ) Eo 2 (EE [aae p
(80)
17

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
where we have used the fact that (<()) 1. The covariance Σg of y governed by the ODE
d = - s) (Ú. -,,)
(81)
+ - {), i.yy, (2IT-, 1]] + 3(T - 8)
(82)
( - () - D) (.) - 1T
(83)
+BL- =) i.) (Ú. (I) T -]] + 3(T -) 5)
(84)
El-3(T s)(2E7- [). -A. - +
3(T s)
(85)
-3((T- - -2 -)E, + 3(T -8)
(86)
= -T - s] diag(
( 1-a(1-))) ()). ) - - B(T - s). Σ
(87)
This motivates a similar transition matrix (note that we multiplied 2 to Φ now)
y (s, so) = Q diag( C 11 -0)) Q
(88)
= Q diag( [0)= (()=-1)a((T0022+11
s)) ([N,-I]a(U~=0)++11 )
(89)
ao)- Σ D
(90)
x)²
which produces the solution
Σ. = (ss, 0) En + So 1 (8 - do
(91)
a(T Σ En 2 En + ΣT a(T-s)7 Σ E7
a(T)²
a(T)²
(92)
Er + oll -) 21 8 E 2Zo ΣTi
a(T)²
(93)
where the integral in the first line is calculated as
So v O))(T - σ) do
(94)
Sh Q diagg a0T
(oa--1)((T-))4111 Q 3(T - d) do
(95)
Q diag( (() () -))+++1 diag(So
So ((2, (A.-I)a(T-a)21) do do)
(T
(96)
s)²
diag(
(On - 1)a(T (5) +1)²
+1)
diag/ (An=1)a(T-s}441
a(T-a)²
Q T
(97)
a(T s))
(A. (7)
a(T-s)² - (a(T
2 Σ ET- a(T)²ET))
(98)
Er- - -T ΣT
xT) D2
(99)
Setting s = = Tkki in Eq. (93), we see that
Cov(ao] = Cov Covlrraall=
= ETA. -wa + aaTa Tanp
a(Tap)? 2
Trip Main-10 Taep ΣT (Covluol - ΣTTr)
(100)
= Eo + aTTxip)) 222 ET Tai
(Cov ( Erery
(101)
This concludes the proof.
B.3. Proof of Corollary 3.3 Corollary 3.3. Suppose Eo = ofI and ΣTAr = I,, and define the quantity (if it exists)
k = (02 (172-1)/(08 - 1).
18

---

Boost-and-Skip: A Simple Guidance-Frig Diffusion for Minority Generation
The variance-ammliffcatiin effect of Eg occurs iff
0
if S S oo. if1 70,0,
(a - (x), 00)
Tatip E
00,a-(x))
if > 7,00,
10)
if I 1 S y and (0o, 7) # 1,
where we define a - (x:) == when K 1.
Proof. Under our assumption, variance--mpllifcatton occurs iff
722 - 1>> (TTs (o) - 1).
(102)
Case 7 S S do. Suppose some value of Taip offers variance amplification. Then
02 0272 - 1>
( skip 2a) (o) 1) 2 0
(103)
which is contradiction.
Case 1 < 7, 0o. In this case, Eq. (102) is equivalent
a (Taip)
(104)
which is equivalent Taip E (a (x), oo) since is a monotone decreasing function. Case 7; do. In this case, Eq. (102) is equivalent
(Tski)) > k
(105)
which is equivalent to Tapp E [0, a (I:)) since a is a monotone decreasing function.
Case do $ S and (oo,1) 7) * the case dg < 1 S
2 a (TTxkp))
(106)
for all values of Taip Likewise, if oo < 11<- < %
- 1 0 5 (TTakip) 1)
(107)
for all values of Txip-
B.4. Proof of Proposition 3.4
Proposition 3.4. Consider an empirical version of the discrete VP-SDE Eq. (8):
{ x. + ( 1)} + v1 0,2
T4-1
Va.
where E NN.. Assume the optimal score function, i.e.. s0(xx V (z)), which trained on an arbitrary New where Naip < N, which are initialized with distinct distributions: EN ~ NO0 I) and &N -N(0,~2I) Assuming that { such that |æc,1l2 < B (as Xu et (2023)), the expected error between samples from these two trajectories at step i {0. (0..... Naip 13 is given by:
data distribution Consider two sample trajectories { and (
N is a bounded process
20
Bi|a
2: <
2(NNo (1) + 7d)..
(108)
X2
where À denotes the contraction rate:
1 1- (ij1 1 ag
A max
Vaj
(109)
jE(itI.....N Noc
and = d(1 ..Nay))
19

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
Proof. The proof is based the contraction theorem for discrete stochastic processes (Pham, 2008). However, unlike the original setup considered in Pham (2008), our focused trajectories start from distinct initial distributions and terminal timesteps: NN ~ (O, I) and Nap (N I). To address this, we leave out { N= and focus errors between (= Nav and { Naw
LN
Employing Nkki as the terminal timestep, the discrete contraction theorem (i.e., Theorem 1 in Pham (2008)) gives the following error bound between and i;; at time i E {0,.... Nakip 1}:
2C + 1 DEll 2 Naa 112)-
B x 2:1l2 S 1 X2
(110)
where == d(1 - (.Na)) Here À indicates the contraction rate that determines the speed error-decaying
1 - āÿ-1
A
max (E+ Naip
Vaj
(111)
1 Cij
Next, we upper-bound the terminal error [|[ x New
@ Naap [1:] as:
E Nap
Ner [12]
E x Napp I1
E Ell Nap [12]
2E 22 Nap 2 Naip
(112)
= E 2 Naup
[2
+ @ Naw
(113)
- 8 + d2,
(114)
where the second equality is due the independent initialization & Naw 2(0,721) The inequality comes from the bounded assumption |2;||2 < B. Now plugging the inequality in Eq. (114) into Eq. (110) yields the desired result:
2C
E[ -z; 12]
+ 2
1aappp00p0 B B
Nuw
(115)
1 1- X2
20
(() 722
(116)
1 X2
(117)
This completes the proof.
B.5. Proof of Proposition A.1
Proposition A.1. Consider the same data distribution po and the optimal score function sa(x, as Proposition 3.2. Suppose the PF-ODE in Eq. (19) is initialized with (T (PTa ETTD Then, the resulting distribution t given by (bDE(()) (0) ~ MPo.ODE: Eo,ODE), where
Po,ODE = p40 + E0 20
1/22 1/2 (00112((UTTu))
(iTTen (Taa)
(118)
Σo,ODE Eg - DTA
(119)
Here, T and Σr are defined as Eqs. (15) and (16), respectively.
Proof. By Lemma B.1, the score function is given
V x (x)))
Va log Nog Ex) -E m.)
(120)
such that the PF-ODE is
dē. = -- (((()), - (o(t)\logp..(e dt
(121)
1-1 + 1,()),,1111)))00))))))),)) (a: a (t)) dt
(122)
(3(1)(E))) 1) dt 31)()) E, 14 dt.
(123)
20

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
with change of variables (Ú., 8) (#T-r, -(), l),
dÿ. [/ - - ds +
1)
- H ds
(124)
ja(T -s) s)Q diag( a(T-s)()-1)) 11) Q y ds
(125)
+ =3a(T -))(TT u diag( T+alT (Ta(T-s)( (1, 1Q My ds.
(126)
Define
6(x; a) = 1/2 log(az² + +1)
(127)
such that
dō(a(T -s))nn)))
(o)((T-))>--1) da(T dST-s) da(T-2) d(T-2) ds
(128)
da
I+a(T ()
-1) ( (T s)) (-1)
(129)
2 - 3 5)
a(T-s)(A,,])
(130)
This motivates the transition matrix
T (s, 8n) = Q diag(e (a - s))dnn 1)- -1-
(131)
Q diag( ((u.-1)a(T-o) RQ
(132)
V(n ()
1/2 - --20
1/2
(133)
with which calculate the solution
ú, (s, + So -0 o diag(Taa(
ItalT 1 (I) QQ" po do
(134)
>ト ΣТ 1/2 ÿo + pT - -1 ET 1/2 HT
HT
(135)
-T +
(00 uT)
(136)
where the integral in the first line is calculated as
So 2 V (s, (T GTT - -q diag( I+a(d (o 'Q_ Mo do
(137)
= Q diag( / (An - () - s)² + 1) diag(
[ 2 a 0T - σ) (ojo(T-o)222,, 1))378 do) d QUT No
(138)
= Q diag( / (An - 1)a(T-)Ta - =]² + 1) diag(
aoT
Q Mg
(139)
v (An 1)l(T [1a[--o)++]
Q diag( / (Am 1)a(T - =)² + 1) diag(
An a(T
V( () ) T Mo
(140)
= a(T - 3)140 - aaTTTT))) 1/2 =// Σ 1/2 440
(141)
PT-s 1/// ΣT 1/2 1/2 pl..
(142)
Setting s = T = Tskip in Eq. (136),
io ÚTAP
M Tua
1/2 1/2
Tap
Tap Taa
11/2 M
(143)
= #40 + 1//2 - Taap
-1/2 (iTra - MTan)
(144)
1/2 1/2 (B (B(arrau)) (T ) + E 122 112 (
Tap
I3 E((rray))
(145)
follows that if TÕ Gaussian do is also Gaussian with
E[Øo] = 44 + 2 "
T Tai
1/2 (12(EEEzz [Bl (Tan))
(146)
Cov En T Cov (
(147)
This concludes the proof.
21

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
B.6. Proof of Proposition A.2
Proposition A.2. Assume the same setup as Theorem in Xu et al. (2023), i.e. || logpr(>))l S L1 and | < any D, in the support of Pt. and reverse-SDE trajectories. Let
f/r
I miin
SDE
SDE (mum mmax mmii), Dimi
SDE y frmas Imax (min))
(148)
2 fmin
ODE
ODE (DE tmse max + fmin), y Itmin ODE tmaxx fmax - tmin)
(149)
where SDE and ODE denote solutions to the reverse- SDE and PF-ODE, respectively. Then
T
tain Ut
ODE ODE)
TV
(mise y Vtmax
(150)
Ta SDE Emin (omn) < (1 (1-20) 2Q 2va((mas)) aa
e BL1/Amin Laannn-Leeee (a Piain
TV(Ptuus Vomas) (151)
where P( ~ 1),
Proof. The first equality follows by noting that the total variation distance is invariant under bijective maps due to the data processing inequality. We now prove the second inequality. Suppose 2; follows the reverse SDE, and consider the change of variables 3; = wt/a((). Then, by the Itô formula,
dæ, = (- -+3(t)a(tt)(-a(())))aa dt a(t) -
(152)
(- (== + =(00-=5(((V(1ogx(())))+++++11
B(t) (0))),(((),,,,,))))))))))) -0000000000000000000000000110000 (og) + /B(T) dwn,)
(153)
dwe,
(154)
a (t)) dwo,
(155)
= (a((t) 3y V lo dt ((((t)= dw,
(156)
where Pt is the density of xt (t(a(t) for az ~ Pt. and we have used the change-of-variables formula of probability density functions the fourth line. Next, using the change of time variable att) -2, we get
dāz = v Pog (1) (z)) di dwy.
(157)
Then following an analogous process as the proof of Lemma 5 in Xu et al. (2023), we obtain the incquality
TV TV Dimi.. S (1 11-2Q 2Q
B
BL1/Cmin L /max fman
T D Dinox)
(158)
2 fmax Emin
where Emin = (tmin)) -2 Emax = a(tmaxx
2 Since the map y, + y (a(s)/a(()) is bijective, we again have
TV( Dimn TV Urmma ) TV Vimes) TV( Y Utmaae)
(159)
(160)
by the data processing inequality. Plugging this relation into Eq. (158) yields the contraction result for the reverse SDE.
C. Implementation Details
Pretrained models. The pretrained model for CelebA was constructed by ourselves by following the settings in Um et al. (2023). The models for LSUN-Bedrooms and ImageNet were taken from the checkpoints given by Dhariwal Nichol (2021). For the results on ImageNet 256 256. we respect the approaches in Sehwag et al. (2022); Um al. (2023); Um Ye (2024b) and leverage the upscaling model developed in Dhariwal & Nichol (2021).
Baselines. The BigGAN model is based upon the same architecture used in Choi al. (2020)3, and we follow the training setup outlined in the official project page of BigGAN'. The StyleGAN results were obtained via the pretrained model
hhttps: thub. com wtt./oo////e//eeeee http://rmooooo////eeee fairgen *htpps: //gi thub. wtw....o//abbooeeeee/e/ Beppocck/iigCA-----
22

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
offered by Karras al. (2019)². The ADM (Dhariwal & Nichol, 2021) baselines used in our experiments employed the same pretrained models those used in our framework, which is provided in the authors' codebase The results of LDM (Rombach et al., 2022) were obtained from the checkpoint offered by Rombach et al. (2022)'. For the EDM (Karras et al., 2022) baseline, we used the checkpoint given in the official project page of (Karras et al., 2022)*. The DiT (Peebles & Xie, 2022) baseline employed the pretrained model provided in the official code repository". The authors of CADS (Sadat et al., 2023) do not provide the official codebase, so we implemented by ourselves based upon the pseudocode provided in the manuscript (Sadat et al., 2023).
The implementation of Schwag et al. (2022) is based upon the descriptions provided in their original paper, employing the same pretrained models ours (i.e., the ADM checkpoints (Dhariwal & Nichol, 2021)). Specifically on CelebA, we created an
(OOD) classifier to distinguish whether an input belongs to CelebA or other datasets (e.g., ImageNet). We then incorporated the negative log-likelihood gradient of the classifier (targeting the in-distribution class) into ancestral sampling to yield low-density guidance. For implementing Um et al. (2023), we followed the settings described in their paper for all datasets under consideration. Specifically, using the same ADM pretrained models as ours, we applied U-Net encoders for minority classifiers and utilized the entire training set to build the classifiers, except for the minority predictor on LSUN-Bedrooms, where only 10% of the training set was used. For the ImageNet 256 x 256 results of Um et al. (2023), we used the upscaling model (Dhariwal & Nichol, 2021) described in Um et al. (2023).
In the additional CelebA baseline with classifier guidance targeting minority annotations (i.e., ADM-ML in Table 1), we respected the same settings outlined in Um et al. (2023); Um & Ye (2024b). In particular, the classifier was trained to predict four minority attributes: (i) "Bald"; (ii) "Eyeglasses": (iii) "Mustache"; (iv) "Wearing Hat". During inference, we generated samples by combining random selections of these four attributes (e.g., bald but not wearing glasses) using the classifier guidance. The backbone model for ADM-ML was the same as ours. The results of Um & Ye (2024b) is based upon their the official codebase respecting their recommended settings reported in the original paper (Um & Ye, 2024b). The temperature sampling results were obtained from the method proposed in (Dhariwal & Nichol, 2021) (in Section G), ie., by using r-scaled score model sø (z, f)/T with 7 = 1.01.
Evaluation metrics. To compute the Local Outlier Factor (LOF) (Breunig et al., 2000) for the generated samples, we used the PyOD implementation (Zhao et al., 2019)" The number of nearest neighbors for calculating AvgkNN and LOF was set to and 20, respectively, as these are commonly used values in practice. Following the approach in Sehwag et al. (2022); Um et al. (2023), both AvgkNN and LOF were computed in the feature space of ResNet-50. The Rarity Score (Han et al., 2022) was computed with k: using the implementation available the official project page
Clean Fréchet Inception Distance (cFID) (Parmar et al., 2022) was evaluated using the official implementation Spatial FID (sFID) (Nash et al., 2021) was computed based on the official PyTorch FID code (Heusel et al., 2017) with modifications to utilize spatial features (i.e., the first 7 channels from the intermediate m xed_6/ 6 conv feature maps), rather than the standard pool.3 inception features. The results for Improved Precision & Recall (Kynkäänniemi et al., 2019) were obtained with A = using the official codebase from Han et al. (2022). To evaluate the proximity to low-likelihood instances at the data tail, we used the least probable instances as baseline real data for computing the quality metries. Specifically for CelebA, we selected the 10,000 real samples with the highest AvgkNN values. For used the most unique 50,000 samples, which had the highest AvgkNN values, as baseline real data. All quality and diversity metrics were computed using 30,000 generated samples.
and ImageNet, we
Hyperparameters. Our hyperparameter selection (7, A₁) followed a two-step approach: first, we determined an appropriate A that ensures a non-negligible a(Txip) (where Taki == T - A,), and then we performed a grid search to select % We empirically found that our framework not that sensitive to the choice of A., and in practice, setting A, such that a (TTxip) > 0.01 generally yields strong performance on low-resolution datasets (e.g., CelebA and ImageNet 64 x 64). For
htt htt.::///gghhtt.cc//////////gggs
23

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
high-resolution benchmarks (e.g., LSUN-Bedrooms), lower threshold of a(Tssi)) 0.005 sufficient, as these datasets sensitive noise intensity (Nichol & Dhariwal, 2021).
For CelebA, we conducted grid search over 7 {4.0, 8.0, 12.0, 16.0, 18.0, 20.0), while for SSUN-Bedrooms, we searched over ?² = {2.0, 4.0, 6.0, 7.0, 7.5, 8.0}. For the ImageNet results, the search was performed over y² (2.0, 4.0, 6.0, 6.5, 7.0, 8.0}. Based on this, we selected the following final values: (i) (7², A,) = (18.0,3) 3) for CelebA; (ii)
(>², = (7.5, 0) for LSSN-Bedrooms; and (iii) (7² A.) = (6.5,3) for the ImageNet cases.
We employed a global setting of 250 timesteps for sampling across all diffusion-basee samplers, including both the baseline methods and approach. For empirical analyses, such the ablation studies, reduced the number of timesteps 100 for efficiency.
Other details. Our implementation is based on PyTorch (Paszke et al., 2019), and experiments were performed on twin NVIDIA A100 GPUs. Code is available at https: /tte:://gtttttt com/ s soobi n- uumuum/eee BB...
D. A dditional Experimental Results
D.I. Neighborhood distance distributions
Figure 7 illustrates neighborhood metric results compared interested four benchmarks. Observe that for all three metrics, approach performs consistently well, rivaling computationally intensive guided minority samplers like Um & Ye (2024b). This highlight the practical significance of Boost-and-Skip, which can achieve great performance improvements in promoting minority features with significantly less computations.
D.2. Additional generated samples
To facilitate more comprehensive qualitative comparison among the samplers, we present extensive set of generated samples across all considered datasets. See Figures 8-11 for details. Observe that samples generated by our approach tend to capture unique dataset features, comparable to those produced by guided sampling techniques. This further highlights the key advantage of our method achieving strong minority generation performance with significantly lower computational costs than existing guidance-baat based approaches (Sehwag al., 2022; Um et al., 2023; Um & Ye, 2024b).
24

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
ADM
ADM-MI.
Soor Um and Yo (2024)
Boost-and-Skin Saindd---ppp-a (Ours)
0.30
0.04
0.25
0.03
Density
0.20
Density
3-
Density
0.02
0.10
0.01
0.05
0.00
0.00
10
1.0
1.2
1.4
1.6
40
60
100
AvgkNN
LOF
Rarity Score
ADM
StyleGAN
DO00
Um and Ye (2024)
Boost-and-Skan (Ours)
0.20
0.020
0.15
0.015
Density
Density
Density
0.10
0.010
0.05
0.005
0.00
0.000
7.5
10.0
12.5
15.0
17.5
20.0
1.0
1.1
1.2
1.3
40
60
80
100
120
140
AvgkNN
LOF
Rarity Score
ADM
Schwag ct al. (2022)
0000
Um and Yo (2024)
Boost-and-SSic (Ours)
0.020
6
0.15
0.015
Densily
Density
Density
0.10
0.010
0.05
0.00
0.000
10
15
20
1.0
1.1
1.2
1.3
1.4
25
50
75
100
125
AvgkNN
LOF
Rarity Score
ADM
Um et al. (2023)
DO0t Um and Yo (2024)
Boost-and-Skip (Ours)
0.20
0.15
0.010
Density
Density
Density
0.008
3-
0.10
0.006
2-
0.004
0.05
0.002
0.00
0.000
10
15
20
1.0
1.1
1.2
1.3
1.4
50
100
150
200
AvgkNN
LOF
Rarity
Figure 7: Comparison of neighborhood density distributions across four benchmarks. (Top row) CelebA 64 x 64. (Second row) L.SUN-Bedroome 256 256. (Third row) ImageNet 64 64. (Fourth row) ImageNet 256 256. "AvgkNN" refers Average k-Nearest Neighbor, and "LOF" is Local Outlier Factor (Breunig al., 2000). "Rarity Score" indicates low-density metric proposed Han al. (2022). The higher values, the less likely samples for all three measures.
25

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
(a) ADM (Dhariwal & Nichol, 2021)
(b) Um Ye (2024b)
(c) Boost-and-Skip (ours)
Figure 8: Samples comparison on CelebA 64 x 64.
26

---

Boost-and-Skip. A Simple Guidance-Fres Free Diffusion for Minority Generation
2112
100
(a) ADM (Dhariwal & Nichol, 2021)
(b) (2023)
(c) Boost-and-Skip (ours)
Figure 9: Samples comparison L.SUN-Beroooss 256 256.
27

---

Boost-and-Skipp A Simple Guidance-Frrg Free Diffusion for Minority Generation
FIEES
YAA
AAAAAA
CAEN FIOR
MfREDOLSTE STo
RFIGH
(a) ADM (Dhariwal & Nichol, 2021)
(b) Sehwag et al. (2022)
(c) Boost-and-Skip (ours)
Figure 10: Samples comparison on ImageNet 64 x 64. Generated samples from five classes are exhibited: (i) "Siberian husky"
(top row): (ii) "water tower" (second row); (iii) "strect sign" (third row); (iv) "ice cream" (fourth row); (v) "cheeseburger" (bottom row).
28

---

Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation
(a) ADM (Dhariwal & Nichol, 2021)
(b) Temperature sampling
(c) Boost-and- (ours)
Figure 11: Samples comparison on ImageNet 256 256. Generated samples from five classes are exhibited: (i) "hamster"
(top row); (ii) "zebra" (second row); (iii) "space shuttle" (third row); (iv) "valley" (fourth row); (v) "volcano" (bottom row).
29


Reviews:
[{"review_id": "bx4C2kU9VA", "reviewer": "Reviewer_bs5F", "summary": "The paper provides two techniques for improving minority sampling. The first technique is a boost, which initialises the sampling with controllable variance. The second technique is skip where it will skip several sampling timesteps. The authors claim to achieve better performance in generating minority samples.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer bs5F for the constructive feedback. Below we provide point-by-point responses on your questions and concerns.\n\n---\n> **1. [bs5F] expressed a concern that the performance is often limited compared to baselines.**\n\nWe note that the superior baselines in Table 1 (e.g., [1,2]) correspond to guidance-based minority approaches that employ guidance terms to direct inference toward low-density regions. While we acknowledge that our suboptimal performance may stem from the lack of such explicit guidance for minority generation, our method significantly advances the Pareto frontier in the performance-complexity tradeoff (see Fig. 1). In particular, our framework delivers notable computational benefits over the guided minority approaches. For example, on ImageNet-64, our method achieves **65% reduction in wall-clock time and 4.5× lower peak memory usage** compared to the current state-of-the-art [2] (see Table 3).\n\n---\n\n> **2. Clarifications on Fig. 5.**\n\nWe would like to assure the reviewer that the visual attributes of our samples in Fig. 5 capture distinctive features of minority instances. For instance, jack-o’-lantern images with bright surroundings (rather than the typical dark Halloween atmosphere) are considered as low-density features of the class [3]. Also, our eagle-class images in the same figure exhibit more intricate visual details compared to the baselines, which are also known as minority features [1,2].\n\n---\n\n> **3. [bs5F] noted missing neighborhood metrics like AvgkNN.**\n\nAs noted in L356-358 (right column), the evaluation results using AvgkNN, LOF, and Rarity Score are provided in Appendix D.1, where B&S performs consistently well across all three metrics, rivaling the state-of-the-art guided minority sampler [2]. See Fig. 7 therein for explicit details.\n\n---\n\n> **4. [bs5F] noted missing comparisons with guidance-based minority methods.**\n\n\nWe would like to gently remind the reviewer that, in Tables 1,3,4, we already compare B&S with guidance-based methods such as ADM-ML [1], Minority Guidance [1], and Self-guidance for minority generation [2].\n \n---\n\n**References**\n\n[1] Don’t Play Favorites: Minority Guidance for Diffusion Models, ICLR 2024\n\n[2] Self-Guided Generation of Minority Samples Using Diffusion Models, ECCV 2024\n\n[3] Generating High-Fidelity Data from Low-Density Regions Using Diffusion Models, CVPR 2022", "claims_and_evidence": "The evidence for achieving a minority sample by the proposed method is not very clear both qualitatively and quantitatively", "methods_and_evaluation_criteria": "1. The method is straightforward and intuitive\n\n2. The improvement is mainly compared with Temperature sampling. Most of the time, the performance is poorer on the baselines (Table 1). Please add more explanation\n\n3. It is hard to judge Figure 5. Can not tell why boost-and-skip provides better minority samples\n\n4. Lack of clear metrics to measure minority samples. In the paper, the authors mention AvgkNN, LOF and Rarity Score, yet all the tables do not have these figures.", "theoretical_claims": "I checked the theoretical claims but not sure if the proofs are correct.", "experimental_designs_or_analyses": "experimental designs are okey", "relation_to_broader_scientific_literature": "n/a", "essential_references_not_discussed": "The work does not compare with guidance methods to show why this method is better than guidance.", "other_strengths_and_weaknesses": "n/a", "other_comments_or_suggestions": "n/a", "questions_for_authors": "1. The improvement is mainly compared with Temperature sampling. Most of the time, the performance is poorer on the baselines (Table 1). Please add more explanation\n\n2. The work does not compare with guidance methods to show why this method is better than guidance. Please include more comparisons.\n\n3. Add more metrics about the minority sample. Current metrics in the paper does not show how minority samples are covered.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rzIMmNWXSy", "reviewer": "Reviewer_7UiG", "summary": "The paper proposes an approach called Boost-and-Skip for generating minority samples using diffusion models. Specifically, it begins stochastic generation with variance-boosted noise to encourage initializations in low-density regions. It then skips several of the earliest timesteps to further amplify the impact of low-density initialization. The effectiveness of Boost-and-Skip is supported by both theoretical and empirical evidence, with the added advantage of low computational cost.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for your detailed comments and valuable suggestions. Below, we provide thorough point-by-point responses to address your concerns.\n\n---\n\n> **1. [7UiG] expressed a concern on the sensitivity to hyperparameters.**\n\nPlease refer to our response to Reviewer o8Xg (the second bullet point).\n\n---\n\n> **2. [7UiG] suggested comparisons with GAN-based minority generation frameworks.**\n\nTo reflect your comment, we have conducted new experiments to compare our method with GAN-based minority generation frameworks. Specifically, we evaluate a class-balancing GAN approach [1] and compare its performance in generating minority samples with ours on CIFAR10-LT (a long-tailed version of CIFAR-10). See the table below for the results.\n\n| Method  | cFID  | sFID  | Prec | Rec |\n|---------|-------|-------|-----------|--------|\n| **DDPM** | 75.71 | 44.26 | 0.95      | 0.23   |\n| **CBGAN**| 78.62 | 43.76 | **0.99**      | 0.08   |\n| **BnS**  | **70.12** | **43.73** | 0.91      | **0.34**   |\n\nAs in the experiments presented in our paper, we used real minority data (from CIFAR10-LT) as the reference for computing the metrics. Observe that B&S outperforms the GAN-based approach in [1] (i.e., CBGAN) even under this highly-biased benchmark, further highlighting its effectiveness as a minority generator.\n\n---\n\n> **3. [7UiG] pointed out that literature reviews on minority-conditional diffusion models are missing.**\n\nWe kindly remind the reviewer that approaches with minority-conditional diffusion models are discussed in the related work section in Appendix A.1. We will move them to the main body in our revision.\n\n---\n\n> **4. Clarifications on Fig. 7.**\n\nWe kindly remind the reviewer that the focus of minority generation is not to replicate the training data distribution but to intentionally bias generation toward minority instances, which are defined as low-density on-manifold samples. In this regard, Fig. 7 demonstrates that our framework effectively achieves this goal, as the high-valued neighborhood metric values imply the generation of low-density instances.\n\nWhile the reviewer might be concerned that high neighborhood metric values indicate the presence of off-manifold (i.e., OOD) samples of poor quality, we emphasize that our method does not generate more OOD samples than state-of-the-art minority samplers such as Minority Guidance [2]. This is evidenced by our superior FID scores compared to Minority Guidance (e.g., in Table 1), where FID is computed using real minority (i.e., on-manifold, low-density) data.\n\n---\n\n> **5. [7UiG] suggested a sanity check of Proposition 3.2 using the toy data in Fig. 2.**\n\nPer your suggestion, we did an experiment for the sanity check in Fig. 1 (provided in the link below). Specifically, we plot generated data variance as a function of initial Gaussian noise variance for the two rings example. Blue dotted line denotes generated data variance predicted by theory, and the orange solid line illustrates actual generated data variance. The general trend of the two curves are similar, validating the ability of B&S to generate minority samples. We conjecture that the offset between theory and practice may arise from the score estimation error, as we use learned scores rather than exact scores to simulate B&S.\n\n- Link to Fig. 1: https://docs.google.com/presentation/d/1GCZKcxbX_A7e_v_ckVCefcVxGsbm2UkZSxyUc5zuSbA/edit?usp=sharing\n\n---\n\n> **6. [7UiG] questioned the effectiveness of our approach on highly imbalanced benchmarks.**\n\nTo address your question, we consider CIFAR10-LT, a highly-imbalanced version of CIFAR-10, and explore the performance benefit of ours. We found that B&S yields improved minority generation even under this biased setting; see the table above (included in the second bullet point of this response) for detailed results.\n\n---\n\n> **7. [7UiG] suggested comparisons with minority-conditional diffusion frameworks.**\n\nWe gently remind the reviewer that our experiments already encompass such approaches, by incorporating ADM-ML [2] - a classifier-guided diffusion sampler conditioned on known minority labels in CelebA. See details in Table 1.\n\n---\n\n**References**\n\n[1] Class Balancing GAN with a Classifier in the Loop, UAI 2021\n\n[2] Don’t Play Favorites: Minority Guidance for Diffusion Models, ICLR 2024", "claims_and_evidence": "The claims are clear and well supported by theoretical and empirical evidence.", "methods_and_evaluation_criteria": "The proposed Boost-and-Skip is quite simple, yet its advantage lies in its rigorous theoretical support.\n\nSome issues:\n\n1. I feel that the proposed method is sensitive to hyperparameters, requiring a wide range of grid searches. Moreover, the optimal hyperparameter settings vary significantly across different datasets. Table 2(c) presents the substantial differences in results under different hyperparameter settings.\n\n2. The authors consider baselines beyond diffusion models. I think some GAN works specifically focused on minority generation should be considered, rather than general-purpose GAN works.", "theoretical_claims": "I did not check the proof very carefully. But their theoretical claims look reasonable.", "experimental_designs_or_analyses": "The empirical study is extensive, covering multiple benchmark datasets and baselines.", "relation_to_broader_scientific_literature": "The paper proposes a simple approach for generating minority samples using diffusion models, achieving performance comparable to state-of-the-art baselines with lower complexity.", "essential_references_not_discussed": "The literature on conditional diffusion models should be reviewed in the main text of the paper, as this is another important methodological branch for minority generation.", "other_strengths_and_weaknesses": "The writing in this paper is clear and easy to follow.", "other_comments_or_suggestions": "N.A.", "questions_for_authors": "1. From Fig. 7, it appears the proposed method would generate OOD samples (original density0). This could lead to an inaccurate learned distribution. How do you explain this limitation?\n\n2. I think the learned distribution for the toy data (Fig. 2) can be plotted to evaluate how well it corresponds to the theoretical result in Proposition 3.2.\n\n3. I wonder about the effectiveness of the proposed method on extremely imbalanced problems. The setup in Figure 5 does not clearly demonstrate this. I suggest, for example, using the CelebA dataset with 99% young and 1% old faces. Additionally, a comparison with conditional diffusion models, given known conditional labels, should be included.\n\n4. The sensitivity to hyperparameters (Methods And Evaluation Criteria).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T1XDfhfHas", "reviewer": "Reviewer_6VDS", "summary": "The paper proposes Boost-and-Skip, a method to generate low-density, minority samples. The method has two straightforward yet effective modifications to standard diffusion models: (1) variance-boosted initialization, and (ii) timestep skipping during the generative process. The authors provide intuitions, theoretical analysis, and synthetic experiments to motivate these two modifications. Empirically, they show that their method achieves competitive performance compared to state-of-the-art guidance-based methods but at significantly lower computational costs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback and for considering our work for acceptance. We appreciate your time and evaluation. If you have any further suggestions or questions, feel free to let us know. We are more than happy to address any additional points or provide further clarifications as needed.", "claims_and_evidence": "The claims made by the authors (i.e., variance-boosted initialization and timestep skipping) are clearly stated and supported by experiments.", "methods_and_evaluation_criteria": "The proposed variance-boosted initialization and timestep-skipping are straightforward and intuitive. The evaluation criteria (e.g., cFID) and datasets (e.g., CelebA, ImageNet) are standard and appropriate for assessing minority generation performance.", "theoretical_claims": "The paper includes theoretical claims concerning the properties of their method. I checked the claims and found that they help me better understand the intuition behind the method; however, I didn't check the correctness of the claims.", "experimental_designs_or_analyses": "The experimental design is rigorous. The ablation studies demonstrate the necessity and impact of each component of the proposed approach (variance-boosting and timestep skipping).", "relation_to_broader_scientific_literature": "The paper is well related to previous works on diffusion models and minority generation. It clearly addresses the efficiency limitation of existing guidance-based approaches and provides an effective alternative.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O4NBvez3qx", "reviewer": "Reviewer_o8Xg", "summary": "Authors of this work propose a method called \"Boost-and-Skip\" for generating minority samples from low-density regions in a data manifold using diffusion models. This method relies on two key modifications to the standard denoising process: 1) Initializing the reverse process with a higher variance noise (instead of a standard Gaussian noise), and 2) Skipping the early denoising steps. Unlike existing diffusion-based methods for minority generation, \"Boost-and-Skip\" does not rely on expensive guidance procedures increasing the efficiency. In addition, authors demonstrate that the proposed method achieves similar performance on the task of minority generation compared with state-of-the-art methods (diffusion-based and others) while maintaining the overall image quality, requiring no additional computations or modules, and being more efficient.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate Reviewer o8Xg for the strong acceptance and thoughtful feedback. Below, we provide detailed point-by-point responses to address your remaining concerns.\n\n---\n\n> **1. [o8Xg] questioned the effectiveness of our approach on highly biased datasets.**\n\nTo address your concern, we consider CIFAR10-LT, a highly-imbalanced version of CIFAR-10, and investigate the performance benefit of ours. See the table below for the results.\n\n| Method  | cFID  | sFID  | Prec | Rec |\n|---------|-------|-------|-----------|--------|\n| **DDPM** | 75.71 | 44.26 | 0.95      | 0.23   |\n| **CBGAN**| 78.62 | 43.76 | **0.99**      | 0.08   |\n| **BnS**  | **70.12** | **43.73** | 0.91      | **0.34**   |\n\n\"CBGAN\" refers to a class-balancing GAN approach that implements minority generation with minority conditional labels [1]. Similar to the experiments in our paper, we used real minority data from CIFAR10-LT as the reference for calculating the reported metrics. Observe that B&S outperforms the considered baselines (including the GAN-based approach in [1]) under this highly-biased benchmark, further demonstrating the robustness of our framework.\n\n---\n\n> **2. [o8Xg] expressed concerns regarding the sensitivity to hyperparameters.**\n\nAlthough we acknowledge that our framework may be sensitive to hyperparameters — particularly $\\gamma$ — we provide a practical heuristic for their selection in Appendix C (Lines 1250–1251), which streamlines the process of choosing both $\\gamma$ and $\\Delta_t$. Leveraging this approach, a simple one-dimensional grid search over $\\gamma$ is sufficient to identify effective hyperparameters (see Lines 1267–1270 for details).\n\nWe also highlight that the implementation complexity of our method is significantly lower than that of existing guided minority samplers, which often involve numerous design choices. For example, the approach in [2] requires training two separate classifiers with many design options (e.g., classifier architectures), while the method in [3] necessitates the selection of six hyperparameters. In contrast, our framework only requires choosing two hyperparameters, providing substantial practical benefits over the guided minority samplers in [2,3].\n\n---\n\n> **3. [o8Xg] expressed concerns regarding the visual quality of minority samples.**\n\nWe believe there are largely two reasons behind the quality degradation mentioned by the reviewer. First, there is a general trade-off between minority sampling performance and image quality. Second, the base diffusion model itself lacks the ability to generate minority features. We provide further explanation for each hypothesis below.\n \nTo investigate the first hypothesis, since FID and Precision are measured with respect to ground-truth minority samples, we can use FID as a proxy for how close the generated distribution is to the distribution of minority data, and Precision as a proxy for the quality of generated minority samples. In Fig. 2 (provided in the link below), we observe there is generally a trade-off between the two quantities, and B&S provides competitive trade-off performance while achieving a dramatic reduction in inference cost compared to guidance-based minority methods. The reviewer is also directed to Table 2 (c), where we again observe FID vs. Precision trade-off as we adjust boosting strength in B&S.\n \n- Link to Fig. 2: https://docs.google.com/presentation/d/1dsMx8s5kJikQnjv6IQNvk-UyC_DgfU9xJpF-ZkaLweI/edit?usp=sharing\n \nAlso, we hypothesize that in some cases, the base diffusion model may lack the capability to synthesize minority features. For instance, on ImageNet, it is well-known that generating human faces is challenging; see Fig. 7 in [4], Fig. 15 in [5], and Fig. 5 (b) in [2]. We believe that is why human faces in Fig. 11 (c) synthesized by B&S appear unnatural and distorted. Using a better base diffusion model may yield more realistic minority samples.\n\n---\n\n**References**\n\n[1] Class Balancing GAN with a Classifier in the Loop, UAI 2021\n\n[2] Generating High Fidelity Data from Low-density Regions using Diffusion Models, CVPR 2022\n\n[3] Self-Guided Generation of Minority Samples Using Diffusion Models, ECCV 2024\n\n[4] Large Scale GAN Training for High Fidelity Natural Image Synthesis, ICLR 2019\n\n[5] Diffusion Models Beat GANs on Image Synthesis, NeurIPS 2021", "claims_and_evidence": "The main claims are that the proposed method is effective at generating minority samples with diffusion models while being more efficient than guidance-based methods. Both the theoretical and empirical results support these claims.", "methods_and_evaluation_criteria": "Authors followed previous works on the choice of metrics to report. I must admit that I am not an expert on this topic but I believe the reported metrics and evaluations do make sense for the problem of minority generation.", "theoretical_claims": "I skimmed over all the equations included in the main paper and did not find any particular errors. However, I note that I did not do a careful read and I am not fully familiar with the background on some of the methodology details.", "experimental_designs_or_analyses": "I find the experimental design to be coherent with previous works on the task of minority generation. So, I do not see any particular pitfalls here.", "relation_to_broader_scientific_literature": "The minority generation problem is important on its own as it focuses on making generative models more inclusive and fair. Since the proposed method is a simple modification on top of standard diffusion models, it is a practical solution that has potential to be easily applied to various frameworks. I also really like the fact that the method builds on simple adjustments to existing processes. This can show that significant improvements are possible by minor smart changes and can encourage the research community to look for simple and efficient solutions for critical and challenging problems such as fairness.", "essential_references_not_discussed": "No. I found the related work section inclusive.", "other_strengths_and_weaknesses": "Strengths:\n1. Simple methodology that builds on two small modifications to the diffusion process for improving minority generation.\n2. Efficient method that does not require additional components or training.\n3. Achiving SOTA performance while reducing the inference time (as compared to SOTA methods).\n4. Good write-up quality and ease of reading.\n\nWeaknesses:\n1. One of the main limitations of this work stems from its relies on the backbone diffusion model's exisiting biases. If the model has severe biases, then Boost-and-Skip will have limited affectiveness.\n2. In addition, as discussed in Table 2, hyperparameters significantly influence the quality of generated samples. This suggests that effectiveness of the proposed method highly depends on a detailed exploration over the hyperparameter values, limiting the method's reliability.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. Can you discuss your point of view on the weaknesses I listed above and if and how these concerns can be addressed? \n2. As can be seen the additional results included in Section D of appendix, it seems like the quality of generated samples is degraded when adjusting the base diffusion model for improved minority generation. For example, in Figure 8, it is easier to tell that samples in b and c are generated while samples in a are more realistic. Or in Figure 11, some samples are very weird-looking (humans in column c). I wonder if you know how badly can the sample quality be affected by improving the minority generation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a new method, Boost-and-Skip (B&S), for minority sample generation using diffusion models. With only minimal modifications—variance boosting and timestep skipping—the method significantly improves minority generation performance without relying on guidance, while also reducing computational cost.

Reviewers appreciated the theoretical motivation and empirical effectiveness of the approach. The authors provided clear and sufficient responses to the concerns raised. The simplicity and general applicability of the method are also notable strengths.

Overall, this paper presents an effective and efficient approach to minority sample generation and is recommended for acceptance.
Decision: Accept (poster)