HiGS: History-Guided Sampling for Plug-and-Play Enhancement of Diffusion Models
Authors: Seyedmorteza Sadat
Farnood Salehi
Romann M. Weber
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion models
sampling
classifier-free guidance
Primary area: generative models
TL;DR: We propose a training-free method that enhances diffusion generation quality across various sampling budgets and guidance scales by reusing the history of predictions made by the diffusion network.

Abstract: While diffusion models have made remarkable progress in image generation, their outputs can still appear unrealistic and lack fine details, especially when using fewer number of neural function evaluations (NFEs) or lower guidance scales. To address this issue, we propose a novel momentum-based sampling technique, termed history-guided sampling (HiGS), which enhances quality and efficiency of diffusion sampling by integrating recent model predictions into each inference step. Specifically, HiGS leverages the difference between the current prediction and a weighted average of past predictions to steer the sampling process toward more realistic outputs with better details and structure. Our approach introduces practically no additional computation and integrates seamlessly into existing diffusion frameworks, requiring neither extra training nor fine-tuning. Extensive experiments show that HiGS consistently improves image quality across diverse models and architectures and under varying sampling budgets and guidance scales. Moreover, using a pretrained SiT model, HiGS achieves a new state-of-the-art FID of 1.61 for unguided ImageNet generation at 256$\times$256 with only 30 sampling steps (instead of the standard 250). We thus present HiGS as a plug-and-play enhancement to standard diffusion sampling that enables faster generation with higher fidelity.

Published as a conference paper at ICLR 2026
HIGS: HISTORY-GUIDED SAMPLING FOR PLUG-AND- PLAY ENHANCEMENT OF DIFFUSION MODELS
Seyedmorteza Sadat', Farmood Salehi', Romann M. Weber
'ETH Zürich, ResareSoocaarhsSSSuuoos (seyedmorteze Jeinf. eeth..cc ch (farnood. salehi, romann. weber webbereeiiinneyreseechhcomm
ABSTRACT
While diffusion models have made remarkable progress in image generation, their outputs can still appear unrealistic and lack fine details, especially when using fewer number of neural function evaluations (NFEs) or lower guidance scales. To address this issue, propose novel momentum-based sampling technique, termed history-guiee guided sampling (HiGS), which enhances quality and efficiency diffusion sampling by integrating recent model predictions into each inference step. Specifically, HiGS leverages the difference between the current prediction and a weighted average of past predictions steer the sampling process toward more realistic outputs with better details and structure. Our approach introduces practi- cally no additional computation and integrates seamlessly into existing diffusion frameworks, requiring neither extra training nor fine-tuning. Extensive experiments show that HiGS consistently improves image quality across diverse models and architectures and under varying sampling budgets and guidance scales. Moreover, using pretrained SiT model, HiGS achieves new state-of-the-art FID of 1.61 for unguided ImageNet generation at 256x255 with only 30 sampling steps (instead the standard 250). We thus present HiGS as a plug-and-ple andd enhancement to standard diffusion sampling that enables faster generation with higher fidelity.
CFG
HiGS (Ours)
Portrait photo female wi th red hair
Figure 1: Sampling with diffusion models using fewer steps lower guidance scales often results in blurry images with artifacts. We propose HiGS, a novel sampling method that enhances quality and details in generated images under various sampling budgets and guidance scales by leveraging weighted average of the diffusion model's past predictions at each inference step. The results shown generated with Stable Diffusion 3.5 (Esser al., 2024) using 10 steps and guidance scale of 1.2.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021b) are a class of generative models that learn the data distribution by reversing a forward noising process that gradually transforms data points into Gaussian noise. Although in theory, the reverse process should yield high-quality samples from the target distribution, diffusion models can produce outputs that are blurry or lack details due to optimization errors and inaccurate estimations of the data distribution at intermediate time steps. This issue becomes more pronounced when using fewer sampling steps or lower classifier-fre free guidance scales (Karras et al., 2022).
The generation process in diffusion models typically involves multiple neural function evaluations (NFEs), which are computationally expensive, especially for large-scale diffusion models containing billions of parameters (Esser et al., 2024). Reducing the number of NFEs reduces the sampling cost but leads to outputs that lack clarity, detail, and coherent global structures, as seen in Figure 1. Although various sampling methods have been proposed to reduce the required sampling steps (Lu al., 2022; Karras al., 2022), existing samplers still rely relatively high step counts (e.g.. 50) to achieve satisfactory results (Podell et al., 2023). Finding training-free methods that enable generating high-quality samples with fewer NFEs remains an open research question.
In addition to iterative sampling, modern diffusion models use high guidance scales to enhance sample quality and prompt alignment (Podell et al., 2023; Nichol al., 2022). Classifier-free guidance (CFG) (Ho et al., 2021) has proven essential for reducing outliers and improving generation quality (Karras et al., 2024). However, CFG doubles the network forward passes per sampling step, thereby increasing the computational cost of inference, and higher guidance scales often lead to oversaturation and reduced diversity (Sadat et al., 2025a; 2024). Consequently, further research is needed to enhance generation quality when using lower guidance scales combined with fewer NFFs.
In this paper, We investigate the sampling process in diffusion models and propose a training-free, momentum-based modification of inference that consistently improves global structure, detail, and sharpness across different sampling budgets and guidance scales. Inspired by momentum-bast variance reduction in stochastic optimization (Cutkosky & Orabona, 2019). we introduce history- guided sampling (HiGS), a novel method for improving the quality of diffusion models by leveraging the history of predictions made by the network. We demonstrate that this prediction history defines an effective guidance direction for steering the sampling trajectory toward higher- regions of the data distribution, especially under fewer NFEs or lower guidance scales. HiGS enables higher quality outputs and more efficient sampling from pretrained models while also improving the generation quality at lower guidance scales avoid the drawbacks of high CFG scales.
HiGS introduces no additional overhead to the sampling process and can be seamlessly integrated into existing diffusion models and samplers without extra training. Through extensive experiments across range of models and setups (including distilled diffusion models), we show that HiGS consistently improves sample quality, particularly in scenarios involving fewer sampling steps or lower guidance scales. Our results indicate that HiGS achieves higher quality metrics compared to standard sampling methods, establishing it as a universal enhancement for pretrained diffusion models under various sampling budgets and guidance scales. Furthermore, using a pretrained SiT model, HiGS achieves state-of-te the-art art FID of 1.61 for unguided (i.e., without CFG) ImageNet generation at 256x256 while using only 30 sampling steps instead of the standard 250.
RELATED WORK
Score-based diffusion models (Song& & Ermon, 2019; Song al., 2021b; Sohl-Dickstein et al., 2015; Ho et al., 2020) learn complex data distributions by inverting a forward process that gradually adds Gaussian noise to the data. These models have rapidly advanced generative modeling, surpassing prior approaches in both fidelity and diversity (Nichol& & Dhariwal, 2021; Dhariwal & Nichol, 2021). They have demonstrated state-of-the-art performance across a wide range of tasks, including unconditional and conditional image synthesis (Dhariwal & Nichol, 2021; Karras et al., 2022; Yu et al., 2025b; Karras et al., 2024), sex-to-image generation (Podell et al., 2023; Esser et al., 2024: Labs, 2024: Qin et al., 2025), video generation (Blattmann et al., 2023b:a; Bar-Tal et al., 2024; Wan et al., 2025), image-to-image translation (Saharia et al., 2022a; Liu et al., 2023a; Xia et al., 2023), and audio generation (Chen et al., 2021; Huang et al., 2023; Liu al., 2023b; Tian al., 2025).
2

---

Published as a conference paper at ICLR 2026
Since the introduction of DDPM (Ho et al., 2020), the field has seen substantial progress, with improvements spanning network architectures (Hoogeboom et al., 2023; Karras et al., 2023; Peebles & Xie, 2022; Dhariwal & Nichol, 2021), sampling techniques (Song et al., 2021a; Karras et al., 2022: Liu et al., 2022; Lu et al., 2022; Salimans & Ho, 2022), and training strategies (Nichol & Dhariwal, 2021; Karras et al., 2022; Song et al., 2021b; Salimans & Ho, 2022; Rombach al., 2022). Despite these advancements, sampling from diffusion models still require relatively high step counts, and various guidance mechanisms-Suun and classifier-free guidance (Ho & Salimans, ensuring strong prompt alignment (Nichol et al., 2022).
as classifier guidance (Dhariwal & Nichol, 2021)
critical for enhancing image quality and
A recent line of work has focused on developing better ODE solvers for the diffusion sampling process, often combined with improved training techniques to make the sampling ODE more linear (Lu et al., 2022: Karras et al., 2022; Esser et al., 2024). Despite these advances, most state-of-the-art samplers still require a relatively high number of sampling steps (e.g., 50 steps for Stable Diffusion XL (Podell al., 2023)), Another direction of research explores distilling the diffusion model into student network capable of sampling with fewer steps (Salimans & Ho, 2022; Song et al., 2023; Sauer et al., 2024). However, step distillation remains computationally expensive, requiring long training on advanced hardware. We show that HiGS serves as a training- freee method to improve generation quality across various sampling budgets and networks (including distilled diffusion models).
Modern diffusion models often rely on high guidance scales to achieve strong image quality and prompt alignment. However, CFG doubles inference cost, and excessive CFG scales reduce diversity and cause oversaturation (Sadat et al., 2024; 2025a), On the other hand, sampling with lower CFG scales and NFEs avoid these issues but typically yield blurry images lacking fine detail and coherent structure. While several methods mitigate the drawbacks of high CFG scales (Kynkäänniemi et al., 2024; Sadat et al., 2025a; Wang et al., 2024), little attention has been given improving quality under low CFG scales and limited sampling steps. We show that HiGS enhances generation across both low and high CFG regimes, offering benefits under varying CFG scales and sampling budgets.
In summary, sampling from diffusion models is an expensive iterative process that might produce unrealistic outputs under certain configurations, We propose a training-fres method that improves generation quality across different sampling budgets and guidance scales, particularly in low-NFE and low-CFG scenarios. Thus, we present HiGS as a plug-and-play enhancement for diffusion sampling.
BACKGROUND
This section provides a brief overview of diffusion models. Let x (a) denote a data sample, and let f. E [0, 7] represent continuous time. The forward diffusion process gradually corrupts data by adding Gaussian noise: & + ott),, where (N, I) and o(t) defines a monotonically increasing noise schedule, satisfying (0) = and (TT) = o maax >> data- As shown by Karras et al. (2022), this forward process can be described by the following ordinary differential equation (ODE):
dx = (t) Zz, log dz = t V log (z))dt,
(1)
where we choose (t) = t, and Pr is the distribution of noisy data at time t, with Pa Paata and Pr = NNO,, I). If the time-dependent score function Vz, log (zt) is known, one can sample from Pasaa by integrating the ODE (or its stochastic counterpart) in reverse, from f T to t O.
In practice, the score function is approximated using a neural denoiser Da t). trained to recover the clean data æ from the noisy input zt. Conditional generation is supported by extending the denoiser to take an additional input (e.g., class labels or text), resulting in Do To(zz,44 (,y).
Classifier-free guidance (CFG) Classifier-fres guidance (CFG) is a technique for enhancing sample quality during inference by interpolating between unconditional and conditional model predictions (Ho & Salimans, 2022). Given an unconditional prediction De (zz, t), the guided denoiser output at each sampling step is computed as:
(2) where WCFG = 1 corresponds to no guidance, and larger values increase conditioning strength. The unconditional model is typically trained by randomly dropping the condition y with some probability p E 10.1, 0.2) during training. Alternatively, a separate network or the conditional model itself can
Dera(z (z, t, y) = were Dolz. t, y) (wera - Do t).
3

---

Published as a conference paper at ICLR 2026
be used to estimate the unconditional score (Karras et al., 2023; Sadat et al., 2025b). While CFG known to improve perceptual quality, .. it often comes at the cost of increased oversaturation and reduced sample diversity (Sadat al., 2025a; 2024).
4 SAMPLING WITH PREDICTION HISTORY
Let to > .... >> tM be the sampling time grid with M +1 steps. The model prediction time step l is denoted by Do(zt, lk. y). Given a window of size W 2 1, we define the history Ha at step k as the set of past predictions prior &x. i.e., Hs {Do(zt, li he,, for I == (max(0. (0. k k- J. We next show how leveraging this history enhances sampling quality, For simplicity, we define Dz (z.))) = (2 , Lxs y). (z,,) Dø(2t, fx),, and (z = Dera(Zt, lis y) to represent the conditional, unconditional, and CFG outputs, respectively.
4.1 MOTIVATION
We first claim that the Euler sampler for diffusion models can be interpreted as performing stochastic gradient descent (SGD) on a time-van varying energy function. To show this, consider the ODE in Equation (1) and discretization {Lo, ......,,,,)) A single Euler step at time l can written as
2tst1 Zth ttx (x+1) VeL. log Pt. (Zt,)
= 21e
ttx tx+1)
VzL, Esg (2x,);
where the energy function (2, is defined via (z.) exp(-FF((z)) for some normalization constant Z. This shows that each Fuler step in diffusion sampling corresponds to a step of gradient descent on the time-dependent energy (z,) with learning rate (tx 14+1).
Motivated by this new perspective, we argue that we can enhance this gradient estimate to improve the efficiency and quality of the sampling process in diffusion models. One promising approach is to augment the gradient with an additional momentum saaree term similar to STORM (Cutkosky & Orabona, 2019), a variance-reduction method for non-convex optimization. Specifically, given differentiable function f. the momentum term in STORM is defined as V f(z,,) - f(z,,-)) ((z .). which incorporates information from consecutive steps for more stable updates. Applying this to Equation (3), we obtain V Es, (z.,) V (Zt,,,) as the momentum term. Equivalently, this can be seen as incorporating the residual of past score terms or model outputs, since the score function corresponds to the energy gradient. In the following, we further generalize this idea to incorporate multiple past predictions rather than only the previous step (similar to multistep ODE solvers (Atkinson al., 2009)). An alternative perspective is given in Appendix A, where connect classifier-free guidance to gradient ascent on specific objective. In Appendix B, we show that the history terms reduce the Fuler solver's local truncation error from ((h)) to ( where hgg 4 4+1, therehy improving the global error from ((h) to (12) with = max h4p
4.2 HISTORY-GUIDED SAMPLING
Building on the insights discussed above, we propose a novel sampling method for diffusion models, termed -guided sampling (HiGS), which integrates past predictions into the current sampling step. We leverage the set of past predictions H4 at each sampling step & improve generation quality such as sharpness, details and global structure. We detail each step of HiGS below.
Buffer input When the sampling is done with CFG, we can choose to keep track of the CFG-guided predictions Dera (zt,) ) the conditional outputs Dz(2i,) the history. We found that the CFG- guided predictions are more effective in improving the quality of sampling. Thus, we use DcrG(2t.) the inputs H₂ when CFG enabled, leading H₂ == (DFGG(2.)) (it))))
Incorporating the history The next design choice is how to use Hi during sampling. Specifically, we want function g that determines the influence of past predictions on the current step. By generalizing the momentum update nule in STORM, we adopt an EMA-style weighted average that emphasizes recent predictions:
((Ha) Ea(t ----------D-o
(5)
seN.
4

---

Published as a conference paper at ICLR 2026
where (0, 1) is a hyperparameter. This formulation integrates information from history while prioritizing recent steps for more informative guidance. We later show that several alternative definitions of such simple averaging, viable options (sce Appendix E for more details).
Let AD,, Drrc (zt,) ) - g(Hx) denote the guidance term in HiGS. A straightforward strategy for combine this update with the current output, analogous to CFG, via DcFG (2 + WHiGs A Dr, with scale WHIGS- In the following, we introduce several improvements over this naive baseline that we found crucial substantially boost performance. Appendix E presents extensive ablation studies to support various design choices of HiGS.
using Ai, inference
Scheduling the guidance weight In our experiments, the benefits of HiGS were most evident during the early and middle sampling steps. We noticed that as sampling progresses, the update term introduces diminishing improvements and may even cause noisy artifacts. To address this, we use a weight schedule WHIGS (tx) that adapts the scale of the guidance term according to the time step tg. We employ square-rool schedule given by
I miin+
WHiSS (t)
WHIGS
byen
Umin
I S mmax
(6)
Pass - Cyen
>> mmax.
Optional orthogonal projection Additionally, we found that it is sometimes beneficial to project the update vector AD, on DcFG (z+,) and downweight its parallel component to prevent oversat- uration and color artifacts (especially at higher values of WHeGs) (Sadat et al., 2025a). The parallel component in AD,, can be computed via
ADI (ADI,, DcHo(ZZa))
(2FG(
(DCFo(Z(,) DCFG (xte)
and the orthogonal component is computed via AD, Dera(zu.) - AD!. We use (n) AD; + nAD!, as the projected update direction, where E [0, 1] is a hyperparameter. As demonstrated Figure 11 (appendix), the orthogonal projection step can mitigate oversaturation and color artifacts in the generated outputs.
drequency-domain filtering We further observed that using AD,, (1)) as the guidance term gener- ally leads unrealistic color compositions in generations (see Figure 15 in the appendix). To solve this, employ frequency-bann -basedd high-pass filtering using the discrete cosine transform (DCT). Since color composition corresponds to low-frequency contents an image, we apply DCT to the update term AD (1)) and attenuate its low-frequency signals with sigmoid high-pass filter:
(R) Sigmoid(A (R))
(8)
where R = v f2 ff is the radial frequency at coordinates (z, v), R, is the cutoff threshold, and controls the sharpness of the transition. This procedure effectively removes the color shifts, leading to more realistic and visually consistent outputs. Accordingly, the final update rule for HiGS is
DHiGs (z+,) (cFG(Z).)) wHias(t)) iDCT(H DCT((AD; (7))).
EXPERIMENTS
We now provide extensive qualitative and quantitative comparisons between standard sampling and sampling with HiGS to show that HiGS enhances the performance of diffusion models under various setups. Further experiments, ablations, and implementation details are provided in the appendix.
Setup We evaluate our method primarily on text-to-image generation using Stable Diffusion models (Rombach et al., 2022; Podell al., 2023; Esser et al., 2024), and on class-conditional generation with ImageNet (Russakovsky et al., 2015) using DiT-XL/2 (Peebles & Xie, 2022), and SiT-XL REPA (Yu et al., 2025a; Leng et al., 2025). For each model, we employ the default sampling algorithms (e.g., the Euler solver for Stable Diffusion XL) and rely the official pretrained checkpoints and publicly released codebases to ensure alignment with the original implementations. Additional details regarding the experimental configurations and hyperparameters are provided in Appendix F.
5

---

Published as a conference paper at ICLR 2026
CFG
+ HiGS (Ours)
CFG
+ HiGS (Ours)
blue car
ballet dancer next to waterfall
CFG
+ HiGS (Ours)
CFG
+ HiGS (Ours)
pizza
modern glass skyscraper at sunset
Figure 2: Effect of HiGS generated samples using standard sampling setups. Compared with CFG outputs, HiGS produces sharper images with improved structure and fewer artifacts.
Evaluation metrics For text-to-image generation, we primarily use HPSv2 (Wu et al., 2023) as our main quality and prompt alignment metric, as we found it to be the most aligned with human judgment. We also report ImageReward (Xu et al., 2023) and the HPSv2 win rate additional preference scores, along with the CLIP Score (Hessel et al., 2021) for completeness. For class-conditional models, we the Fréchet Inception Distance (FID) (Heusel et al., 2017) the primary metric to assess both image fidelity and diversity, given its strong alignment with human visual perception in this setting. conducted under standardized setup minimize
To ensure fair comparisons, all FID evaluations sensitivity to implementation differences. We also report Inception Score (IS) (Salimans et al., 201 6), Precision (Kynkääänniei et al., 2019), and Recall (Kynkäänniemi et al., 2019) as complementary metrics to separately evaluate sample quality and diversity.
5.1
MAIN RESULTS
Qualitative comparisons We first qualitatively evaluate the impact of HiGS on generation quality under three different regimes: the normal setup using practical CFG scales with high NFEs (Figure 2), sampling with fewer number of NFEs (Figure 3), and sampling under low CFG scales (Figure 4). We note that HiGS is able improve generation quality under all these settings, showing that its benefits cover a wide range of CFG scales and sampling budgets.
HiGS vs CFG scale Next, we quantitatively evaluate the effect of adding HiGS the sampling process on generation quality across different CFG scales WCFG- Figure 5a shows that HiGS is beneficial across all guidance scales, maintaining a significant margin in terms HPS win rate.
HiGS the number of sampling steps Similarly, Figure 5b shows that HiGS outperforms the CFG baseline in generation quality across all NFEs. This demonstrates that HiGS is beneficial for all sampling budgets, and its advantages are not limited to specific number of sampling steps.
Comparison with different models Furthermore, across different models and metrics Tables 1 and 2, using fixed guidance scale and sampling steps per baseline for both sampling with and without HiGS to ensure fair comparison. As before, HiGS consistently improves generation quality across diverse setups and metrics, indicating that serves universal enhancement standard diffusion sampling.
evaluate the impact of HiGS output quality

---

Published as a conference paper at ICLR 2026
CFG
HiGS (Ours)
CFG
HiGS (Ours)
pink dog
A basket macaroons
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Close-up photo of cupcake
Detailed photo of lion
Figure 3: Comparison of standard sampling and HiGS under fewer number NFEs. HiGS outputs exhibit significantly better global structure, sharpness, and details than the baseline.
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Glossy red teapot on reflective surface
Photoreal marble bust, studio render lighting
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Cafe latte in round red cup and saucer
stack of buttermilk pancakes
Figure 4: Illustration of the effect of HiGS on generated outputs under lower guidance scales. HiGS leads to noticeable improvements image quality and details compared to the baseline.
7

---

Published as a conference paper at ICLR 2026
Wis
HPPS22
Win Besk
CIC HIGS
34
02
0
-Sicpe
Sicpe
(a) Changing the guidance scale
(b) Changing the number sampling steps
Figure Effect HiGS across guidance scales and sampling budgets using Stable Diffusion 3 (Esser et al., 2024). HiGS consistently outperforms standard sampling with CFG in all settings, highlighting its effectiveness in improving generation quality under varying guidance scales and sampling budgets.
Table 1:
Quantitative evaluation of the effect of HiGS on sampling. HiGS consistently improves all metrics across different models, demonstrating higher generation quality than the CFG baseline. For fairness, sampling with and without HiGS is performed using the same NFE and CFG scale.
Model
Guidance
FID 4
1
Precision
Recall f
CFG
12.08
187.11
0.68
0.73
SYT-XI REPA (Yu et al., 2025a)
HiGS (Ours)
4.86
277.20
0.80
0.70
CFG
8.73
173.21
0.72
0.68
DiT-XL/2 (Peebles & Xie, 2022)
HiGS (Ours)
7.15
180.05
0.75
0.71
CFG
28.49
35.07
0.56
0.54
Stable Diffusion XL (Podell al., 2023)
+HIGS (Ours)
26.18
36.22
0.59
0.57
CFG
27.19
40.11
0.73
0.41
Stable Diffusion (Esser et al., 2024)
+HiGS (Ours)
26.84
40.94
0.76
0.42
Table 2: Evaluation of using HiGS with various Stable Diffusion models. HiGS improves all metrics reflecting human preference for image quality and prompt alignment across multiple benchmarks. All comparisons conducted with the same NFE and CFG scale for faimess.
Benchmark
Model
Guidance
ImageReward
HPSv2 f
Win Rate
CFG
-0.091
0.224
0.07
Stable Diffusion XI.
HiGGS (Ours)
0.148
0.249
DrawBench (Saharia et al., 2022b)
CFG
0.491
0.257
0.18
Stable Diffusion
HiGS (Ours)
0.621
0.272
CFG
0.621
0.258
0.21
Stable Diffusion 3,5
HiGS (Ours)
0.702
0.270
0.79
0.191
0.239
Stable Diffusion XL
HiGS (Ours)
0.360
0.261
0.92
Parti Prompts (Yu al., 2022)
0.843
0.273
0.19
Stable Diffusion
HiGS (Ours)
0.919
0.285
0.81
CFG
0.879
0.270
0.18
Stable Diffusion 3.5
HiGS (Ours)
0.935
0.282
0.82
CFG
0.327
0.245
Stable Diffusion XI.
HiGS (Ours)
0.515
0.275
0.96
IIPS Prompts et al., 2023)
CFG
0.820
0.279
0.21
Stable Diffusion
HiGS (Ours)
0.901
0.291
0.79
CFG
0.821
0.274
0.18
Stable Diffusion 3.5
HiGS (Ours)
0.889
0.289
0.82
5.2 IMAGENET BENCHMARK
Table 3 shows that HiGS improves the performance of recent ofate-of-the-art models on conditional ImageNet generation at 256x256 resolution. For unguided generation (i.e., without CFG), HiGS
8

---

Published as a conference paper at ICLR 2026
Table 3: Effect of adding HiGS recent state-of-the-art methods for conditional ImageNet generation at 256x256. HiGS significantly improves sampling speed, matching the FID of the original models in just 30 steps. Moreover, HiGS achieves new state-of-the-an FID of 1.61 for conditional generation without CFG using the SiT-XL REPA-E model (Leng al., 2025).
Model
Guidance
# Steps +
FID 4
IS 1
Precision f
Recall f
Unguided
250
1.83
217.30
0.77
0.66
+HiGS (Ours)
30
1.61
240.75
0.81
0.62
REPA-E (Leng et al., 2025)
CFG
250
1.26
314.90
0.79
0.66
+HiGS (Ours)
40
1.32
306.10
0.80
0.65
Unguided
250
5,90
157.80
0.70
0.69
+HiGS (Ours)
40
5.43
165.91
0.71
0.68
REPA (Yu al., 2025a)
CFG
250
1.42
305.70
0.80
0.65
+HiGS (Ours)
40
1.44
306.80
0.80
0.64
Table 4: Effect of adding HiGS to the sampling process of distilled models. HiGS improves the quality of these models, showing that its effects complementary diffusion distillation.
Model
Guidance
ImageReward f
HPSv2 1
Win Rate
CLIP Score
CFG
0.774
0.273
0.03
0.332
SDXL-FFlsh (SD-Community)
+HiGS (Ours)
0.864
0.298
0.97
0.333
CFG
0.63
0.277
0.18
0.318
SDXI.-Lighming (Lin al., 2024)
+HiGS (Ours)
0.66
0.285
0.82
0.317
reduces the state-of-the-ar art FID from 1.83 to 1.61 using only 30 sampling steps, without any retraining of the base model. Moreover, by improving sample quality at lower NFEs, HiGS provides training- free way to accelerate sampling from existing models. For guided generation with CFG, HiGS matches the performance of the base models in just 40 sampling steps, compared 250 steps with the default sampler. Finally, this experiment shows that HiGS is compatible with guidance interval (Kynkäääniemm al., 2024), as both CFG baselines incorporate this technique in their samplers.
5.3
COMPATIBILITY WITH DISTILLED MODELS
Finally, we also demonstrate that HiGS is compatible with distilled diffusion models that use fewer sampling steps. Diffuusio distillation reduces sampling steps by training a student model to replicate the performance of the base model. As shown in Table HiGS enhances the sampling quality of these distilled models, achieving a significant win rate according to the HPS score. This shows that the benefits of HiGS are complementary diffusion distillation, and that HiGS can be applied training-free method to further enhance the quality of distilled diffusion models.
DISCUSSION AND CONCLUSION
In this work, we presented history-guided sampling (HiGS), a simple, training-free modification of diffusion sampling that leverages the history of model predictions to steer the reverse process toward higher quality and more coherent images. By applying history-informed correction that emphasizes the deviation between the current prediction and weighted average of past predictions, HiGS mitigates blur and artifacts that often arise during sampling, particularly with fewer number of NFEs or lower CFG scales. HiGS requires no additional network evaluations, integrates seamlessly with existing samplers and architectures, and is fully plug-and-pla Our experiments demonstrated that HiGS consistently improves perceptual quality and fidelity across diverse models and sampling budgets, with especially strong benefits in low-NFE and low-CFG regimes. While highly effective, HiGS still inherits, albeit to lesser extent, the biases and some limitations of the underlying diffusion models, and addressing these challenges remains promising direction for future work.

---

Published as a conference paper at ICLR 2026
BROADER IMPACT STATEMENT
Our method has the potential to improve the realism and quality of outputs produced by diffusion models without requiring expensive retraining, making it practically valuable for applications in visual content creation. However, as generative modeling technologies continue to evolve, the ease of producing and distributing synthetic misleading content also increases. While such advances significantly boost creativity and productivity, they also raise important ethical and socictal concerns. It is therefore essential to raise awareness about the potential misuse of generative models and to carefully consider the broader societal implications their deployment. For an in-depth discussion ethics and creativity in generative modeling, refer readers to Lin & Losavio (2025).
REPRODUCIBILLIT STATEMENT
Our work builds upon the official implementations of the pretrained models cited in the main text. Algorithm I gives the algorithm for applying HiGS inference, and the pseudocode for HiGS is presented in Algorithms 2 and Additional implementation details, including the hyperparameters used in the main experiments, given in Appendix F.
REFERENCES
Kendall Atkinson, Weimin Han, and David E Stewart. Numerical solution of ordinary differential equations. John Wiley & Sons, 2009.
Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere: space-time diffusion model for video generation. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-11, 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rom- bach. Stable video diffusion: Scaling latent video diffusion models to large datasets. CoRR, abs/2311.15121, 2023a. doi: 10448500/ARXIV2311..55 2311.15127.
URL https://doi.or/// http://ddo.org/10.85555/rr 18.48500/arXivv
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis, Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12563-22573 2023b.
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Waveg- rad: Estimating gradients for waveform generation. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL
Ashok Cutkosky and Francesco Orabona. Momentum based variance reduction in non-convex sgd. Advances in neural information processing systems, 32, 2019.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans image synthe- sis. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jen- nifer Wortman Vaughan (eds.), Advances Neural Information Processing Systems 34: An- nual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 8780 8794, 2021. URL https//poeedinssneurppppccpppppp 19nd23ddee9fis4baB877.ooo 1f0.002268dffffaaaa Abba Abstract
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.03206, 2024.
E. Hairer. S. P. Nørsett, and G. Wanner. Solving ordinary differential equations I (2nd revised. ed.): nonstiff problems. Keringer-Verlage Berlin, Heidelberg, 1993. ISBN 0387566708.
10

---

Published as a conference paper at ICLR 2026
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. CoRR, abs/2104.08718, 2021. URL htrps////aaxr// abs/2104.08716.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vish- wanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6626-6637,, 2017. URL. http::/mwwccdiinssseerrcccc/pppp (tre0000eeoeeef65887760000000700
Abstract.html.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. CoRR, abss:aaa. 2022. doi: 10.48550/u2X1..2207.u 12598. URL
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan Tien Lin (eds.), Advances Neural Information Processing Systems 33: Annual Conference Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL. https: 1aabl11194b---sttttiia Abstract..tml.
Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. CoRR, fass/210..1282,, 2021. URL
https:/aariv.orr//b///16.. 15282.
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. CoRR, abs/2301. 1093, 2023. doi: 10.485500axxi::330..10000 URL 48550/arXiv http://00...rg1100005555500/x1/ https://doi.orr//10..
10093.
Qingqing Huang, Daniel Park, Tao Wang, Timo Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Havnø Frank, Jesse H. Engel, Quoe V. Le, William Chan, and Wei Han. Noise2music: Text-conditioned music generation with diffusion models. CoRR, abs/2302.03917, 2023. doi: 10448550aaxx..22002.0911 URL https://doo.org/10.455553 arXiv.2302.03917.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. 2022. URL http:pppaereeeeeeeeeeeommmmmdd/a
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models, 2023.
Tero Karras, Miik Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with a bad version of itself. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL http:://oeeeeevieewmm/louuummddd
Tuomas Kynkäänniemi. Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Im- proved precision and recall metric for assessing generative models. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Ahh---Bu, Emily B. Fox, and Roman Gar- nett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancou- ver, BC, Canada, pp. 3929-3938, 2019, URL Abstracttttt.....
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. CoRR, abs/2404.07724, 2024. doi: 1044550/ARXX122400007744 URL https://doi.org/
10.44550//rxii.24444.000
Black Forest Labs. Flux.
2024.
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504. 10483, 2025.
11

---

Published as a conference paper at ICLR 2026
Shanchuan Lin. Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation, 2024.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision ECCV 2014 13th European Conference. Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V, volume 8693 of Lecture Notes in Computer Science, pp. 1440-75,, Springer, 2014. doi: 10.1007/ 3-319- 106022-2--
978-3-319- 10602-11 1848. URL httpp::/dooi.org111.000007
Xiaojian Lin and Michael Losavio. A comprehensive survey bias and faimess in generative ai: Legal, ethical, and technical responses. Ethical, and Technical Responses (March 04, 2025), 2025.
Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A. Theodorou, Weili Nic, and Anima Anandkumar. s2sb: Image-to-image schrödinger bridge. CoRR. abs/2302.05872, 2023a. doi: URL
10.485550/rrX1..2222.55555
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. arXiv preprint ar arXiv:2301.1250: 2023b.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL http:pps/pppeeeriieeeeeeffoonnn/ PIKWVdyyBKY.
Cheng Lu, Yuhao Zhou Fan Bao, Jianfei Chen, Chongxuan Li. and Jun Zhu. Dpm- solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In NeurlPS, 2022. URL hatrs5eeeAAsssttteoCooeeeeeemee
260al
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 8162-811 8171. PMLR, 2021. URL http://proceedings.mrr mlr. prrpp 1390micchl22a..tm....
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GL.DE: towards photorealistic image generation and editing with text-guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 16784-16888 PMLR, 2022. URL https://proceedings. mir. v162/nichol22a.html
William Peebles and Saining Xic. Scalable diffusion models with transformers. CoRR, abs/2212.09748, 2022. doi: (0.48550/a2iv..222..7777 URL http:://doi.org/104885555 2212.09748.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis, CoRR, abs/2307.01952 2023. doi: 1044550/AXXIV2307001555 URL https:
Qi Qin, Le Zhuo, Yi Xin, Ruoyi Du, Zhen Li, Bin Fu, Yiting Lu, Jiakang Yuan, Xinyue Li, Dongyang Liu, et al. Lumina-iman image 2.0: A unified and efficient image generative framework. arXiv preprint arXiv:2503.21758. 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 10674-10685. IEEE, 2022. doi: 10. 00L11099CCPP32228222220000 URL https:://oi.org// wt../o/.///ooo/o..oo 1009 01PR52688.20222.111
12

---

Published as a conference paper at ICLR 2026
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis., 115(3):211-22,, 2015. doi: 10.1007/s1 1 -63-015--8....
URL https://doi.orr//10.. 1000///////..0/////// 1263-015-0816-))
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M. Weber. CADS: Unleashing the diversity of diffusion models through International Conference Learning Representations, 2024. URL https://ppentviie.wee//oo net////rowe////mmm id=zMoNrajk2X.
sampling. The Twelfth
Seyedmorteza Sadat, Ounar Hilliges, and Romann M. Weber. Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In The Thirteenth International Conference on Learning Representations, 2025a. URL http://ppneeeiiiio//////oo/ddddd
Seyedmorteza Sadat, Manuel Kansy, Otmar Hilliges, and Romann M. Weber. No training, no problem: Rethinking classifier-free guidance for diffusion models. In The Thirteenth International Confer- ence on Learning Representations, 2025b. URL http://pppeeeeeeeeeeee/o/omcdaca
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Palette: Image-to-imagee diffusion models. In Munkhtsetseg Nandigjav, Niloy Mitra, and Aaron Hertzmann (cds.), SIGGRAPH '22: Special Interest Group Computer Graphics and Interactive Techniques Conference, Vancouver, BC, Canada, August 7- 11, 2022, pp. 15:1-15:10. ACM, 2022a. doi: 10. 00.1145/35282333533077 URL https://doi.org/10 1145/3528233.35337777
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L.. Denton, Seyed Kam- yar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. 2022b. URL http://papers..nip httppssccpppppeaaeeeeeee haappeeellll/pppee2222/ssss
3 -bbttrrttttttttttt--teoo -ww.tract-Coofeeeneeeee Conference.htm
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL http:////eeneeeeeeeeeee//on/t/td
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 2226 2234,, 2016. URL 841363aaee9224428886666161661 00000000077711444000005eeeeeeee1 Abbtraaetht--.sssaa...tttt
6/hash/
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In Takeo Igarashi, Ariel Shamir, and Hao (Richard) Zhang (eds.), SIGGRAPH Asia 2024 Conference Papers, SA 2024, Tokyo, Japan, December 3-6, 2024, pp. 106:1-106:11 11. ACM, 2024. doi: 10.1145/368052833667666 1ttp:////rr.011111388088888888 T1111114//688888888 36687666
URL https://doi.org/// 10.
JD-Community. Sdxl flash in collaboration with project fluently.
https:/hnuggingface.co
sd-community/sdkl
Accessed: 2024-09-08.
Jaseha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics.. 37:2256-2265, 2015. URL http://proceedings. mlt.press/v337/shhi-ddckksss 1.. ht.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria. May 3-7, 2021. OpenReview.net, 2021a. URL
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Bus Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual
13

---

Published as a conference paper at ICLR 2026
Conference Neural Information Processing Systems 2019, NewrIPS 2019. December 8-14, 2019. Vancouver, BC, Canada, pp. 11895-11907, 2019. URL
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria. May 3-7, 2021. OpenReview.net, 2021b. URL http:///oceeiiiooooo/ooooooocaaa
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning. ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 32211- 32252. PMLR, 2023. URL http:///wwcceeiiiin.mmmpss/////p
Zeyue Tian, Yizhu Jin, Zhaoyang Liu, Ruibin Yuan, Xu Tan, Qifeng Chen, Wei Xue, and Yike Guo. Audiox: Diffusion transformer for mmythnngtt--audin generation, 2025.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.2033 2025.
Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. Trans. Mach. Learn. Res., 2024, 2024. URL http:///peeeeeriiie..eee//oonodd
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.088 09341, 2023.
Bin Xia, Yulun Zhang, Shiyin Wang, Yitong Wang, Xinglong Wu, Yapeng Tian, Wenming Yang, Radu Timotfe, and Luc Van Gool. DiffiZi: Efficient diffusion model for image-to-image translation, 2023.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation, Advances in Neural Information Processing Systems, 36:15903-15935, 2023.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Bureu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation. Trans. Mach. Learn. Res., 2022, 2022. URL https///ppenreiee.nnel
foummidd-AFDeYJJIND.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jínwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. In International Conference on Learning Representations, 2025a.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang. Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. 2025b.
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipe: A unified predictor- corrector framework for fast sampling of diffusion models. CoRR, abs/2302.04867, 2023. doi: 10.445550/axXi.222224445 URL
14

---

Published as a conference paper at ICLR 2026
A CFG AS GRADIENT ASCENT
This section provides an alternative intuition behind HiGS. Recall that CFG update at time step t can be written in the alternate form
(zro ) = (zt,)) (wera (Dz (zt.)).
(10)
which can be interpreted as a single gradient ascent step on the objective
(DFG((D(z Du( - Dc(z
- (1
(z
(11)
w.r.t the model output Dz(z,,) ) (Sadat et al., 2025a). Here, sgl.] denotes the stop-gradient operation. This suggests that we can augment the gradient update in CFG with a history-based momentum term similar to STORM (Cutkosky& & Orabona, 2019). For a differentiable function f. STORM uses
the gradient difference V f( V f(zzz-1) to incorporate information from consecutive steps.
Applying this to the CFG objective from Equation (11), we have
v fcra(zz.) = (z(z..) Du(z:
(12)
V ( De( (u(z)))
(13)
so the STORM momentum reduces to
V fcr,( fCFG = De(z).) De(2(e-1)
(14)
This suggests enhancing the guidance direction with history term along Dz(zt.) De De(zt. ). In
Section 4, we generalized this idea to incorporate multiple past predictions rather than only D2(2t2-)) and introduced several refinements that further improve generation quality.
B ERROR ANALYSIS OF HIGS
We now show that adding HiGS to the Euler solver for diffusion models can improve the convergence rate of its local truncation error, and hence leading to better global estimates with fewer sampling steps. Let u( u(z,,t t) = t V log Pr (z)). The sampling ODE for diffusion models is then equal to
dz --(2 Ze, t) The Euler step for this ODE time step t& is equal
2iets = Eic + (t& - tx+1)u($ tx)
(15)
= 2 + h 1k);
(16)
where hu 4x - is the step size, and 2,o -O, 0 m. Now, assume that instead of u( (x). we follow update similar to HiGS based on the previous update u( u(z i.c.,
ū(zte> tx) u(z tx) + wv (u(ztz tk) u( tx-1))
(17)
for some time dependent weight schedule ww = w(tx).
Theorem B.I. Let () be sufficiently smooth in both arguments with bounded derivatives, and let hy = 4 - 14+1 denote the variable step size for time grid to >> 1 >... > Las with M 4 1 steps. Then, at each time step tg, the Euler update in Equation (15) has local truncation error (h In contrast, there exists a weight wa E IR such that the modified update rule of Equation (17) used in HiGS, yields local truncation error (bb)). Consequently, the global error improves from (h)ffe for Euler to (h²) with HiGS, where h max -x+1))
Proof. Let (t) denote the ground truth solution to the sampling ODE (derived by integration). For simplicity, we define z =ttx). Using Taylor expansion, we get
Zk+1 = nz + dz
1 d2z
(1x)((++1 - (x) +
2 di2 (GI) - 4x)²
(18)
dt
for some C1 E [1441, 4g). Since we have dz
di = -u(2, 1), we get
1 ddz
Zo+1 2x - (1x+1 - (x)u(zs, (zx, 4x) +
(G))(4+) - -4)²
(19)
2 di2
1/ du
=2 + huu(ze, (x)
(=(()),())
(20)
2 d
15

---

Published as a conference paper at ICLR 2026
Accordingly, the local truncation error L3 of the euler method is given by
h du
LE
| - (zu + (zu 44) 11))
(=(()))s))
S Ch
(21)
2 di
for some constant C. This shows that the local truncation error for the Euler solver is equal to (
We next show that using the history- update in Equation (17) improves this rate to (h2)). If we use the Taylor expansion of u(zx-1, 11-1), we get
du
h h d'u (z tx) +
u( 1, 1x-1) = u( =( tx) + hk-1 dt
(z((2) (2)
(22)
2
do
for some Ç2 [t,, 1 This leads to ü( tx) u(zk: tx) wkhe- dt
du
wxh
d²u
(z):
tx)
dt (z (S2)
(23)
2
Using another Taylor expansion for zt) (this time up to third order), we get
dæ
dd
xk++ = 21 +
(tx) (tx+1
(x) + 2 dl² (tx)(t++1 - +2) +
6 d (ts) - tx)
(24)
di
1ig du (#k, tx)
h d=u
2k + (xu( th)
(=(<a),
(25)
2 dt
di
for some {3 E [Is+1, 4x]. Accordingly, the new truncation error i is given by Ls ||x+11 - (Zts + h2 (x))
(26)
=
ho hhe(wklik> - hR du
2 dt (z (e) +
hi d'u
6 dr2 Cs) +
wghgh d'u
2 de (=((2)
G2)
(27)
Now, if we set wh- = 2hh
by the first term vanishes, and we get
h d²u
6 dt (=(Cs). Cs) + hihe- d²u
L.
(=((2): G2)
(28)
4 dt
Since the step sizes are bounded. there is constant A such that h S Ahr. We therefore have
duu
Ah2
ddu
L
d (= (=(())))) +
(=((2) 62)
(29)
6
4
dt
Assuming that u(2, t) is sufficiently smooth, its derivatives should be bounded on [ty, tol. Thus.
LA S Ch O(h))
(30)
for some constant C". Therefore, Theorem B.2 implies that for h max tk+1), the global error is O(h) for the Euler update and ( for the history-based update in HiGS.
Theorem B.2. Let u(z,t) t) be sufficiently smooth in both arguments, and consider a decreasing time grid to > t > ... > tus with variable step sizes h2 tt -t+11 Suppose a numerical ODE solver produces updates with local truncation error La of order (hT)), for some p 2 1. Then the final global error of the solver Er, satisfies F.s ((h)) for h max; (1+ -t++1)-
Proof. A general proof is given in Chapter II.3, Theorem 3.4 of Hairer et al. (1993).
C RELATION TO AUTOGUIDANCE
Autoguidance (Karras et al., 2024) leverages weaker diffusion model (a smaller or less trained variant of the base model) to improve generation quality. However, since it requires training an additional model, cannot be applied out of the box enhance pretrained models without extra training. In contrast, the history term introduced by HiGS can be interpreted as an explicit negative signal derived from the previous predictions of the diffusion model. Specifically, the prediction Da(2t,) at each step corresponds to a denoised version of the current input Σt,, which initially tends to be blurry and of lower quality. As sampling progresses, successive predictions become sharper, and thus the update term De(z..) - g(Hs) roughly captures the difference between the base model's s prediction and that of a "worse" version. Because the negative signal g(H(.) is defined entirely from past predictions, HiGS can be applied to pretrained models without any additional training. Moreover, HiGS can be combined with autoguidance by replacing the CFG prediction Dera (2),, in HiGS with the corresponding prediction from autoguidance.
16

---

Published as a conference paper at ICLR 2026
APG
+ HiGS (Ours)
APG
HiGS (Ours)
APG
+ HiGS (Ours)
APG
HiGS (Ours)
Figure 6: Compatibility of HiGS with APG (Sadat et al., 2025a) on Stable Diffusion 3 (Esser et al., 2024). HiGS enhances the quality and detail of APG, indicating that its benefits are complementary to various guidance methods.
D ADDITIONAL EXPERIMENTS
Sampling time Since HiGS does not require additional forward pass, its sampling time is effectively identical to the CFG baseline. To verify this, we measured inference performance on Stable Diffusion using the same GPU, with and without HiGS. In both cases, sampling achieved about 6.50 iterations per second with identical memory usage, confirming that HiGS has negligible impact runtime memory.
Compatibility with adaptive projected guidance We next demonstrate that HiGS is compatible with other CFG variants such as adaptive projected guidance (APG) (Sadat et al., 2025a). Figure shows that HiGS also improves the quality and details of the APG samples. This indicates that our approach is complementary to various CFG modifications.
CLIP scores While HPSv2 and Rewaed both account for image quality and prompt alignment, we also report CLIP scores in this section for completeness. Table 5 shows that HiGS achieves the same CLIP score the baseline, indicating that preserves the prompt alignment of CFG. We note, however, that CLIP scores may not fully reflect human judgment, as they often show limited variation across models (Podell al., 2023). Therefore, we primarily relied on HPSv2 as our main metric for evaluating both quality and prompt alignment.
Compatibility with different samplers The main results of this paper examined the effect of adding HiGS to various models with their default samplers. In this experiment, explieitly demonstrate that HiGS is compatible with different diffusion sampling techniques using DiT-XL/22 as the base model. Table shows that HiGS improves the quality across all samplers (including multistep solvers such as DPM++), indicating that its benefits are complementary to changing the sampling method.
Avoiding the issues high CFG seales While increasing CFG generally improves image quality at various NFEs, high CFG scales often result in oversaturation and reduced diversity. In contrast, HiGS enhances image quality at lower CFG scales while inherently avoiding these drawbacks Figures and 8 demonstrate that applying HiGS at lower CFG scales yields more diverse and realistic generations compared to sampling with high CFG. Furthermore, as shown in the main paper, HiGS consistently outperforms CFG across wide range of scales.
17

---

Published as a conference paper at ICLR 2026
Table 5: Comparison of CLIP scores across datasets and models. HiGS consistently improves generation quality (see Table 2) while maintaining virtually identical CLIP scores. This demonstrates that the quality gains are achieved without sacrificing prompt alignment.
Benchmark
Model
Guidance
CLIP Score
CFG
0.308
Stable Diffusion XL
+HiGS (Ours)
0.309
DrawBench (Saharia et al., 2022b)
CFG
0.329
Stable Diffusion 3
+HiGS (Ours)
0.327
CFG
0.334
Stable Diffusion 3.5
+HiGS (Ours)
0.330
CFG
0.317
Stable Diffusion XL
+HiGS (Ours)
0.318
Parti Prompts (Yu et al., 2022)
CFG
0.327
Stable Diffusion 3
+HiGS (Ours)
0.325
CFG
0.330
Stable Diffusion 3.5
+HiGS (Ours)
0.329
CFG
0.333
Stable Diffusion XL
+HiGS (Ours)
0.336
HPS Prompts (Wu et al., 2023)
CFG
0.332
Stable Diffusion
+HiGS (Ours)
0.330
CFG
0.337
Stable Diffusion 3.5
+HiGS (Ours)
0.336
Table 6: Effect of adding HiGS to various popular diffusion samplers using the DiT-XL/2 model with 15 steps and WCFG 1.25. Note that HiGS improves the performance all samplers (including multistep solvers such as DPM++), and hence its effect is complementary to changing the sampler.
Sampler
Guidance
FID +
IS f
Precision +
Recall
CFG
11.87
151.40
0.69
0.70
DDIM (Song et al., 2021a)
+HiGS (Ours)
8.73
173.21
0.73
0.69
CFG
7.15
180.05
0.75
0.71
DPM+ (Lu et al., 2022)
+HiGS (Ours)
6.66
191.45
0.76
0.70
CFG
24.65
107.25
0.58
0.67
DDPM (Ho al., 2020)
+HiGS (Ours)
17.03
128.38
0.64
0.64
CFG
6.75
183.11
0.74
0.72
PLMS (Liu al., 2022)
+HiGS (Ours)
6.13
191.44
0.75
0.72
CFG
6.79
185.86
0.75
0.71
UniPC (Zhao et al., 2023)
+HiGS (Ours)
6.60
192.81
0.76
0.69
Changing the scale in HiGS Figure 9 shows the performance of HiGS as the scale WHiGS varies. We observe that performance improves as WinGs increases, but degrades when the scale becomes too large, while all settings still outperform the CFG baseline. Overall, we find that WInGS & 3 provides consistently strong performance across models.
E ABLATION STUDIES
The choice for buffer input Table shows that both the conditional prediction De(z.) and the guided prediction (era(z,) are viable inputs to HiGS, but using DCrG as the history leads to better performance. Accordingly, we used Dera(z,) (2,) in experiments whenever possible.
18

---

Published as a conference paper at ICLR 2026
High CFG
Low CFG + HiGS (Ours)
High CFG
Low CFG + HiGS (Ours)
Figure 7: High CFG scales improve overall structure of the image but lead to lack of diversity, oversaturation, and unrealistic effects. HiGS significantly improves the quality of lower CFG scales, leading to more realistic and diverse generations. Samples are generated with Flux (Labs, 2024).
Frequency filtering Figure 10 shows that applying frequency filtering is essential for avoiding color artifacts in our method. Without this step (both with and without projection), the images often exhibit unrealistic patterns and unnatural color compositions. These issues are effectively mitigated through our DCT-based filtering, and later show that performance remains fairly robust different choices of hyperparameters used in the DCT filter.
Projection We next demonstrate that projection can also reduce color artifacts the generated images in some situations. Figure 11 shows that after projection, the generations appear realistic with fewer oversaturated regions. Thus, incorporating projection into HiGS may lead to more realistic outputs when oversaturation exists.
19

---

Published as a conference paper at ICLR 2026
High CFG
Low CFG + HiGS (Ours)
High CFG
Low CFG + HiGS (Ours)
Figure 8: High CFG scales improve overall structure of the image but lead to lack of diversity, oversaturation, and unrealistic effects. HiGS significantly improves the quality of lower CFG scales, leading to more realistic and diverse generations. Samples are generated with Flux (Labs, 2024).
Effect of the thresholds in the weight schedule We next analyze the impact of fmin and fmax in the weight scheduler. Figure 12 shows that setting fmin too low or too high leads to either excessive insufficient guidance, resulting in suboptimal performance. Similarly, Figure 13 indicates that reducing tmax generally causes insufficient guidance and degraded results. Overall, we found that setting min (0.3, 0.5] and mmax [0.9. 1.0] yields consistently good performance across models.
Effect of EMA value Figure 14 shows that HiGS performs well for wide range of EMA values We have found that setting a 0.5 or a = 0.75 gives good results across all models and architectures.
20

---

Published as a conference paper at ICLR 2026
HPSv2
ImageReward
CLIP Score
0.4
0.00
0.23
01:02
0.25
0.20
CFO
CRG
CFG
0.13
11:22
Figure 9: Effect of varying WHiGS on different quality metrics with Stable Diffusion XL (Podell et al., 2023). HiGS consistently outperforms the CFG baseline across a wide range of WHiGS scales. In practice, we found that setting WHiGS I generally leads to good performance across models.
Table 7: Comparison of using the conditional model prediction vs the CFG prediction as input HiGS. While both options outperform baseline sampling without HiGS, using the CFG-guided prediction (when available) leads to better results.
Config
HPSv2 f
Image Reward 1 CLIP Score t
Baseline (with CFG)
0.238
0.174
0.317
+HiGS (Conditional)
0.249
0.234
0.315
+HiGS (CFG)
0.255
0.371
0.322
w/o DCT filtering
w/ DCT filtering
w/o DCT filtering
w/ DCT filtering
Figure 10: Effect of frequency filtering on generation quality. Without DCT filtering, the results show unrealistic color compositions, which corrected after applying the filtering operation.
w/o projection
w/ projection
w/o projection
w/ projection
Figure 11: Effect of using projection in HiGS. Without projection, the generations might produce oversaturated results, which can be mitigated by reducing the strength the parallel component.
21

---

Published as a conference paper at ICLR 2026
HPSv2
ImageReward
CLIP Score
0.3
11.22
0.33
0.25
0.2
0.31
02
Figure 12: Effect of varying min on different quality metrics for Stable Diffusion XL. The results show that performance degrades when miin is set low too high. We find that min E [0.3, 0.5] yields good results across all models.
ImageReward
Score
0.50
0.3
0.95
0.20
0.5
0.0
0.7
01.5 11.0
01.5
0.00
10.7
0.9
0..
0.6
6.7
0.5
Figure 13: Effect of varying mmax on different quality metries for Stable Diffusion XL. The results show that reducing max often leads to insufficient guidance and degraded quality. We recommend setting max E [0.9, 1] for all models.
Effect of the DCT threshold We show in this section that HiGS is relatively robust to the choice of the DCT threshold R. as long as it sufficiently low. Figure 15 shows that the metrics are more or less the same for lower R. values. We set R, x 0.05 for all models.
Effect of weight scheduling We next show that alternative choices for the weight scheduler are possible. Specifically, we test a constant function for winGs (t) applied over an interval, i.e.,
t Σ fmin
WHIGS (t)
WHIGS
tamin < t S tmax
(31)
t fmax:
as well linear schedule given by
S 1min:
WHiGS
WHiGS
Emin
< t S t
(32)
bamn
0
tmaxx
Table 8 shows that all three options are viable. We chose the square-root function in Section 4, empirically found that it produces more visually appealing results.
Various options for the history function Table 9, we show that different choices of g yield similar quality metrics, indicating that HiGS is robust to this design decision. We adopt the EMA option, as it produced slightly more realistic outputs in our visual evaluations. Moreover, it enables computing the average the fly without storing all past predictions in memory (see Algorithm 2).
22

---

Published as a conference paper at ICLR 2026
HPSv2
ImageReward
CLIP Score
043
0.54
0.
0.24
0.23
-0.5
0.16
11:00
O
Figure 14: Effect of varying the EMA parameter a on different quality metries for Stable Diffusion XL. The results indicate that performance remains consistent across choices of a.
HPSv2
ImageRoward
CLIP Score
0.30
0.32
0.
01:08
6.18
5-3012
D- 0-30-2
00.30
5-10""
DCT threshold
DCT threshold
DCT threshold
Figure 15: Effect of varying the DCT threshold different quality metrics for Stable Diffusion XI.. The results indicate that performance remains consistent across different threshold choices.
Table 8: Effect of different weight schedulers on HiGS.
Weight Scheduler
WHIGS
HPSv2 f
Image Reward 1
CLIP Score
Constant
1.75
0.261
0.36
0.319
Square-root
2.50
0.261
0.39
0.319
Linear
3.25
0.260
0.37
0.318
Table 9: Effect of different averaging functions g(Ha) on HiGS.
g (H+))
HPSv2
Image Reward
CLIP Score
Random
0.261
0.362
0.318
Average
0.261
0.349
0.319
Weighted average
0.260
0.370
0.320
EMA average
0.255
0.371
0.322
F IMPLEMENTATION DETAILS
The detailed algorithmic procedure for HiGS is provided in Algorithm 1, with pseudocode given in Algorithms and Since HiGS reuses past predictions without requiring additional forward passes, its computational cost equivalent standard CFG. Other operations, such as DCT, add negligible overhead. Thus, HiGS improves quality without increasing the overall sampling cost or memory as can be seen in the code and our experiments.
We always scale the time step such that E [0, 1]. For the frequency filter, we found that = 50 and R. 0.05 work well across all models. Similarly, setting /min E [0.3, 0.5) and mmax E [0.9, 1.0) lead
23

---

Published as a conference paper at ICLR 2026
Table 10: Guidance parameters used for Table
Model
Steps
WCFG
WHIGS
T
Emin
max
a
Re
SiT-XL + REPA
30
1.5
0.3
0.75
0.05
DiT-XL/2
15
1.25
2
0
0.3
1
0.75
0.05
Stable Diffusion XL
20
2.5
1.75
0.4
0.75
0.05
Stable Diffusion 3
20
2.5
1.75
0
0.4
0.75
0.05
Table 11: Guidance parameters used for Table
Model
# Steps
WCFG
WHIGS
Emin
tmax
Re
Stable Diffusion XL
20
2.5
1.75
0.4
0.5
0.05
Stable Diffusion 3
20
2.5
1.75
0.4
0,5
0.05
Stable Diffusion 3.5
20
2.5
1.75
0.4
0.5
0.05
Table 12: Guidance parameters used for Table 3.
Model
# Steps
WCFG
WHIGS
Emin
fmax
Re
SiT-XL REPA (Unguided)
40
1
0.4
1
0.75
0.05
SiT-XL REPA (with CFG)
40
1.8
1
0.35
1
0.75
0.05
SiT-XL REPA-E (Unguided)
30
1.25
1
0.35
1
0.75
0.05
SiT-XL REPA-E E (with CFG)
40
2.5
0.75
0
0.3
1
0.75
0.05
consistently strong results in our For EMA values, observed that 0.5 or 0.75
worked well across all experiments. We used all past predictions as the history length W, which allows compute the EMA average on the fly without buffering all previous predictions in memory. The hyperparameters used for each experiment are given in Tables 10 to 12.
For quantitative evaluation, FID scores for class-conditional models in Table I reported using 10,000 generated samples and the full ImageNet training set. The ImageNet results in Table 3 are based on 50,000 generated samples to ensure fair comparison with prior work. For text-to-image models, we used the entire validation set of the COCO 2017 dataset (Lin et al., 2014) as ground truth text-image pairs. We followed the ADM evaluation framework (Dhariwal & Nichol, 2021) for computing FID, IS, Precision, and Recall to maintain consistency across all evaluations.
G ADDITIONAL VISUAL EXAMPLES
We provide additional visual examples in this section to demonstrate the effectiveness of HiGS in enhancing the quality of various models across a wide range guidance scales and sampling setups. Figures 16 to 19 show further text-to-image generation results using Stable Diffusion models (Podell et al., 2023; Esser et al., 2024). In addition, Figure 20 presents visual results for applying HiGS to Flux (Labs, 2024). Finally, Figure 21 provides consiionniiiinal generation with SiT-XL REPA (Yu et al., 2025a). In all cases, HiGS consistently improves quality over the baseline.
24

---

Published as a conference paper at ICLR 2026
Algorithm Sampling with HiGS
Require: Diffusion model Dg. input condition
Require: CFG scale WCKG. HiGS schedule WHiGs (1), history length W
Require: EMA parameter a, projection weight 1). DCT parameters (Rey A)
1: Initialize latent 2T (O, I), history buffer H +- 0
2: for ly (lo, 4....,,44 (T) do
3: Compute conditional and unconditional predictions: De( ), Du(zs))
4: Apply CFG: Pcra(zt,) = wera Dz(2t,) (wera D1) 1.D..(z). (zee)
5: Compute the history signal g(Hs)) = Lier, a(1 - -a)=-1-
1- DCFG((1.)
6: Form the guidance direction: AD = DCro - g(H())
7: if projection enabled (i.e., < 1) then Project ADr. into orthogonal and parallel components w.r.t DeRa (z..)
8:
9:
AD + AD, +AD!!
10:
end if
11:
Apply DCT-based high-pass filter: ADt, + iDCT(H DCT(AD)..)
12:
Apply HiGS: DHIGS (zt,) DCFG (Zz,) WHiGS(t2, (t& A
13:
Update history: H4+11 H4 J (DCFG (z4)). }}, truncate oldest if [H++1] > W
14:
Apply one sampling step: - = SAMPLINGSTEP( Diaas (zt), 21,
15:
end for
return 20
25

---

Published as a conference paper at ICLR 2026
Algorithm 2 Utility functions used in the implementation of HiGS.
import torch import torch_dct dct
def project(
v0: torch. Tensor, # [B, C, W]
vl: torch. Tensor, # [B, C, H, W]
dtype = vê. dtype
vo, v1 double(), v1.double()
v1 = torch. nn. Functional normalize dim=[-1, -2, -3])
vê_parallel (ve v1) sum(sdimmdmm1)) -2, -3], keepdim=True) v1
vê_orthogonal = ve v@_parallel
return vē_parallel. to(dtype), v0_or vo_orrhogonal. ttthoooal.toodttyeee
def square_root_schedeeee
start=0, end=1):
if t end or t start:
return 0.0 return ((t - start) (end start)) ** 0.5
def dct2(x):
return dct.dct_2d(x, norm="ortho")
def ix)t2(x):
return det. idcct_dcxx... norm="ortho")
def apply_high__ss ler.frrg_dttmssssdiii.... threshold=0.05, sharpness=50):
C, H, W diff.
device = diff.devicc
dct2(diff)
u torch. (H, device=device) view(H, 1)
v torch. (H, device=device) view(1, W)
d = torch. sqrt(u + v**2) # normalized distance from top-left (DC)
mask torch. sigmoid((d threshold) sharpness)
x_filtered = X mask # broadcast over (B, C)
diff_filtered idct2 (xft(tx_ilteeeed toddiff.dtype)
return diff_filtered
class HistoryBuffer:
def _ini___nt___ttt (self, ema_alpha=0.11 75):
self. ema = None
eelf.ema_alpha ema_alpha
def add(s,lf, current_pred))
if self ema is None:
self.eml ema torch. zeros_ pegros_likeccurrent__reee
self. ema self. ema_alpha current_pred (1 self. ema_alpha) self.em ema
26

---

Published as a conference paper at ICLR 2026
Algorithm PyTorch implementation of HiGS.
class HiGSGuidance:
def _init__
self,
w_higs,
t_min=0.44 4,
t_max=1.00 .0,
eta=1.0,
ema_alpha=0. 75,
det_threeholl==..055
):
self. history bestoryBurferr(ma_allheeeeee
self. weight = w_higs
self.min_t t_min
self malftmaxtt> t_max
wwr.ppaalllleeee weight eta self self tehf.dlltthrrehhlld act_thheshold
def step(self, current_pred, timestep-None):
Compute the HiGS guidance step.
w..
current_ppeat can be either CFG-guided or conditiona predictions
if self. wistoristrr...oo ema is None:
self hii..histoy..eo www.hitooo..dddcrrreee.. add(current_ppre) return zeros. weweerrllleeeeeeeeeeeeeeeeeeeeee (cuurentr))tt_prree)
diff = current_pred self www.hhittteee...
# compute the projection of the difference diff_par, diff_orth projectddiff, current_pred) diff diff_orth diff_par self paa.paahllle
Compute the scaling factor based the current timestep gamma seaarre_roo_scheuleeetteee scale self weight gamma
min_t,, self max_t)
Update the history with the current prediction self. history.adddcureeetl add(current__rre)
# Apply the high-frequency DCT mask to the difference
self wet.dddtttreshol... diff htapyyhhigh.free__ttttetseeee
chreshold=self.dee wttpdddsslll.d..e.eee.e tct_shhreshold
Return the scaled difference return scale diff
27

---

Published as a conference paper at ICLR 2026
CFG
HiGS (Ours)
CFG
HiGS (Ours)
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Figure 16: Generated samples using Stable Diffusion XL with 20 steps and werG 3.
CFG
HiGS (Ours)
CFG
HiGS (Ours)
Hell WOrLD
HELLO WORLD
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Figure 17: Generated samples using Stable Diffusion 3 with 16 steps and WCFG = 2.
28

---

Published as a conference paper at ICLR 2026
CFG
HiGS (Ours)
CFG
HiGS (Ours)
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Figure 18: Generated samples using Stable Diffusion 3 with 28 steps and WCFG 4.5.
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
HOT
HISTORY
CFG
+ HiGS (Ours)
CFG
+ HiGS (Ours)
Figure 19: Generated samples using Stable Diffusion 3 with 28 steps and WCFG = 2.
29

---

Published as a conference paper at ICLR 2026
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
OPEN 24 HOURS
OPEN 2 4 HOURS
CFG
+ HiGS (Ours)
CFG
HiGS (Ours)
Figure 20: Generated samples using Flux with 15 steps and WCFG = 1.25.
30

---

Published as a conference paper at ICLR 2026
CFG
HiGS (Ours)
CFG
HiGS (Ours)
CFG
HiGS (Ours)
CFG
HiGS (Ours)
Figure 21: Class-conditional generation with SiT-XL + REPA using 30 steps and WCFG = 1.8.
31


Reviews:
[{"review_id": "RV9Va60TnS", "reviewer": "Reviewer_BPak", "summary": "This paper proposes History-Guided Sampling (HiGS), a training-free method that improves diffusion model sampling quality by leveraging past model predictions. Inspired by momentum-based variance reduction in optimization (STORM), HiGS computes a weighted average of previous predictions and uses the difference from the current prediction as guidance. The method integrates seamlessly into existing samplers without additional forward passes, requiring only hyperparameter tuning. Extensive experiments across multiple models (SD-XL, SD3, DiT, SiT, Flux) and settings demonstrate consistent improvements, with particularly strong results in low-NFE scenarios. Using SiT-XL+REPA-E, HiGS achieves FID 1.61 on ImageNet 256×256 with just 30 steps (vs. 250 baseline), establishing new state-of-the-art for unguided generation.", "questions": "1. **Hyperparameter selection**: For practitioners applying HiGS to a new model, can you provide: (a) A simple default configuration that works reasonably well across models? (b) A recommended tuning protocol (e.g., \"start with w_HiGS=1.5, α=0.75, then adjust based on visual quality\")? (c) Sensitivity analysis showing how much performance degrades with suboptimal choices?\n2. **High-resolution overhead**: What is the computational overhead of DCT/iDCT at high resolutions (1024×1024, 2048×2048)? Can you provide wall-clock time comparisons showing actual latency impact in production settings?\n3. **AutoGuidance comparison**: How does HiGS compare quantitatively with AutoGuidance (Karras et al., 2024) in terms of quality, speed, and applicability? Could HiGS and AutoGuidance be combined?\n4. **Diversity metrics**: Table 1 shows slight Recall decrease for SiT-XL+REPA (0.73→0.70). Can you provide additional diversity metrics (e.g., Vendi score, cluster-based metrics) to quantify diversity preservation? Are the diversity benefits claimed in Figures 7-8 statistically validated?\n5. **DCT filtering justification**: Why was DCT chosen specifically for frequency filtering? (a) Have you compared with other methods? (b) Figure 15 shows Rc=0 performs reasonably well—is DCT essential for metric improvements, or primarily for visual quality? (c) What is the quantitative impact of DCT on metrics beyond FID? (d) Why you choose frequency filtering as the solution of “unrealistic color compositions in generations” issue?\n6. **Advanced momentum methods**: While STORM provides the initial motivation, recent advances in momentum-based optimization (e.g., Lion, SOAP) suggest promising extensions. Do you see these as promising future directions, and what challenges would they present for the training-free nature of HiGS?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Strong practical value**: Training-free and truly plug-and-play (Algorithm 1), addressing a critical need for faster diffusion sampling. No additional forward passes required (Section D confirms identical 6.50 iter/sec with/without HiGS).\n2. **Comprehensive experimental validation**: Extensive testing across architectures (U-Net, Transformer), model scales (675M-2.6B parameters), training paradigms (standard, distilled), and tasks (text-to-image, class-conditional). Table 6 demonstrates compatibility with multiple samplers (DDIM, DPM++, PLMS, UniPC).\n3. **Consistent improvements**: Win rates > 0.75 across most settings (Table 2), with particularly strong results in low-NFE scenarios (Table 1: SiT-XL+REPA FID 12.08→4.86, -60%). Figure 5 shows benefits across all CFG scales (1.5-5.0) and NFE ranges (10-30 steps).\n4. **State-of-the-art achievement**: ImageNet 256×256 unguided FID 1.61 with only 30 steps (Table 3) represents significant advancement in fast, high-quality generation.\n5. **Well-motivated methodology**: Clear connection to optimization theory (Eqs. 3-4) and formal error analysis (Theorem B.1). Thorough ablation studies justify design choices (Tables 7-9, Figures 10-15).\n6. **Compatibility with existing methods**: Works with distilled models (Table 4), guidance interval (Table 3 baselines), and can combine with APG (Figure 6).", "weaknesses": "1. **Statistical significance not established**: All experiments appear to use single runs without error bars or confidence intervals. Some improvements are small (e.g., Table 1 SD3: 27.19→26.84, only -1.3%) and may not be statistically significant. Multiple seeds and significance tests would strengthen claims, especially for marginal improvements.\n2. **Hyperparameter selection guidance insufficient**: Tables 10-12 show different hyperparameters per model (w_HiGS ranges 1.0-2.5, η varies 0-1, t_min 0.3-0.4) without clear selection principles. For practitioners applying HiGS to new models, the paper lacks a practical tuning protocol or recommended defaults. Figures 12-13 show t_min sensitivity (~13% performance variation), contradicting \"robust\" claims.\n3. **Limited theoretical-practical connection**: Theorem B.1 suggests optimal weight w_k = h_k/(2h_{k-1}), but implementation uses EMA with α=0.5-0.75. This gap is not explained. Additionally, orthogonal projection and DCT filtering lack principled justification—they appear to be heuristic solutions found through trial-and-error rather than theoretically motivated.\n4. **Incomplete comparison with related work**: No direct comparison with AutoGuidance (Karras et al., 2024), which is conceptually similar but requires training. Appendix C mentions compatibility but provides no quantitative results. Comparison with other training-free fast sampling methods would contextualize the contribution better.\n5. **Computational overhead not fully quantified**: While Section D reports identical iteration rates, this is for SD3 at unspecified resolution. DCT/iDCT operations have non-negligible cost at high resolutions (e.g., 1024×1024 or 2048×2048). Wall-clock time measurements across resolutions and batch sizes would be valuable.\n6. **Missing failure case analysis**: Section 6 mentions \"biases and limitations\" but provides no concrete examples. When does HiGS not help or potentially harm? Honest reporting of failure modes would aid practical deployment decisions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YMIr9Ndwiu", "reviewer": "Reviewer_zxKC", "summary": "The paper proposes History-Guided Sampling, a training-free, momentum-style modification to diffusion sampling. It injects a correction term at each step using an EMA of past denoiser predictions. The update is scheduled over time, optionally projected to suppress the parallel component to avoid oversaturation, and high-pass filtered in the DCT domain to prevent color drift. The authors motivate HiGS by viewing Euler sampling as SGD on a time-varying energy and borrowing a STORM-like momentum term, as well as offering an error analysis showing improved local truncation error and thus better global error order for the modified update. The experiments show some improvements of HiGS compared with Stable Diffusion variants, DiT/SiT with low NFE and low CFG. It is notable that the unguided HiGS reports FID 1.61 in 30 steps on ImageNet-256.", "questions": "1.\tThe error analysis (App. B) shows an existence result for a weight $w_k=\\frac{h_k}{2 h_{k-1}}$ that upgrades local error to $O(h_k^3)$. In practice the paper use an EMA history, a scheduled $w_{\\text {HiGS }}(t)$, projection, and DCT filtering. Can the author quantify how closely the implemented EMA + schedule approximates the ideal $w_k$ ? A small toy ODE or linearized score model comparison would help.\n2.\tSection 4.1 frames Euler as SGD on a time-varying energy and connects the history term to STORM style gradient differences. Can the authors provide an empirical plot of $\\left\\|\\nabla E_t\\left(z_t\\right)-\\nabla E_{t-1}\\left(z_{t-1}\\right)\\right\\|$ vs. the EMA residual magnitude during sampling to validate the link?\n3.\tThe theorem assumes smoothness and bounded derivatives. Are there empirical signs (e.g., at extreme CFG scales) that these assumptions break and the history term can destabilize?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper introduces a training-free “history” term during sampling (EMA over past predictions) and combines it with two stabilizers—orthogonal projection and DCT high-pass filtering. The combination is simple in this plug-and-play form, and well-motivated by both optimization (STORM-style momentum) and numerical analysis views.\n2.\tThe paper gives two theoretical lenses: (1) Euler sampling ≈ SGD on a time-varying energy, motivating history/momentum; (2) a truncation-error analysis showing that a properly weighted history term can improve the local error from O(h^2) to O(h^3) and thus the global error order. \n3.\tThe experiments show consistent gains across models including SDXL/SD3/SD3.5, DiT/SiT, tasks, and multi samplers. Quantitative tables show large preference-metric jumps and lower FID/IS steps to match baselines; ImageNet results include a new unguided FID 1.61 in 30 steps on SiT-XL+REPA-E", "weaknesses": "1.\tThere may be a gap between the clean theory and the implemented recipe. The error-order analysis assumes an ideal history weight $w_k$ and doesn't account for EMA history, time-windowing, orthogonal projection, or DCT filtering used in practice; the proof shows existence of a good $w_k$, not that the EMA-driven schedule approximates it. \n2.\tText-to-image improvements are primarily via HPSv2/ImageReward/Win-rate; CLIP stays flat in Table 5. There is no formal human evaluation or statistical significance tests across prompts or seeds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jBn5U37LJK", "reviewer": "Reviewer_etRw", "summary": "This paper introduces History-Guided Sampling (HiGS), a novel, momentum-based sampling technique designed to enhance the image quality and sampling efficiency of existing diffusion models. The core insight is that integrating the history of recent model predictions into the current inference step can steer the process toward more realistic outputs with improved fine details, particularly when using a low number of neural function evaluations (NFEs) or lower classifier-free guidance (CFG) scales. The method leverages the difference between the current noise prediction and a weighted average of past predictions. Crucially, HiGS operates as a plug-and-play enhancement, requiring no additional training or fine-tuning of the base diffusion model, and introduces negligible computational overhead. The authors demonstrate HiGS's effectiveness across various models (including Stable Diffusion 3 and SiT) and achieve a new state-of-the-art FID of 1.61 using a pretrained SiT model, showcasing both strong efficiency and quality gains.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "1. High Utility and Plug-and-Play Nature: The primary strength is that HiGS is a highly practical and post-hoc sampling enhancement. It requires no modifications to the pre-trained diffusion model or additional training, making it universally applicable and easy to integrate into existing workflows.\n\n2. Exceptional Efficiency and Quality: The method boosts image quality even under highly constrained inference budgets (low NFEs). By leveraging a simple weighted average of history predictions, the approach introduces practically no additional computational cost.\n\n3. Effectiveness: HiGS is shown to be effective across diverse models and architectures (SD3, SiT, SDXL-Flash), indicating that the concept of \"history-guided\" momentum is a fundamental enhancement to the general sampling process, rather than a solution tailored to one specific model.", "weaknesses": "1. Lack of Novelty and Differentiation from Prior Work (UCGM [1]): The approach bears significant conceptual similarity to prior work, specifically UCGM (see [1]). The paper needs to provide a rigorous comparison (both theoretical and empirical) to clearly articulate how HiGS fundamentally differs from and improves upon UCGM's methodology, particularly in its formulation of the history/momentum term.\n\n2. Inconsistency in Theoretical Formulation and Experiments: The theoretical foundation is established using the EDM (Elucidated Diffusion Models) formulation, yet the experimental evaluation is conducted on models trained using different frameworks (DDPM for DiT/SDXL, Flow Matching for SiT/SD3). This mismatch between the theoretical background and the experimental setup must be reconciled. The authors should either generalize the theory to cover the different diffusion formulations used or provide an EDM-based experimental validation.\n\n3. Weak or Misleading Theoretical Justification (SGD/Energy): The introduction of concepts related to Stochastic Gradient Descent (SGD) and energy functions to justify the momentum term (e.g., logic around lines 186-187) appears tenuous and possibly forced (\"shoehorned\"). The authors must critically re-evaluate and simplify the theoretical derivation to establish a clear, direct, and convincing link between the proposed history-guided mechanism and the underlying physics/optimization, or remove the irrelevant theoretical connections.\n\n4. Insufficient Interaction Analysis in Ablation Study: The ablation studies (e.g., Tables 7, 8, and 9) are presented linearly and fail to adequately analyze the interaction between the various design components. It is unclear how designs are built upon each other (e.g., whether Table 8 is based on Table 7, or how $g(H_k)$ in Table 9 integrates with prior designs). The authors must clarify the baseline for each subsequent ablation and provide more comprehensive, possibly combinatorial, results to demonstrate the synergistic or interdependent effects of their design choices.\n\n5. Theoretical Justification of the Momentum Term: While the mechanism is clearly defined, the paper should provide a deeper theoretical analysis or intuition explaining why the divergence from a historical average (the \"difference between the current prediction and a weighted average of past predictions\") systematically leads to higher quality, more detailed results. Is it effectively smoothing the variance of the noise prediction, or is it introducing a form of implicit guidance toward the average manifold learned during training?\n\n6. Hyperparameter Sensitivity Analysis: As a momentum-based technique, HiGS inherently introduces a set of new parameters (e.g., the weighting factor for the history, the depth or length of the history used). The paper needs to include a thorough sensitivity analysis demonstrating how robust the results are to changes in these hyperparameters. Are the reported gains dependent on a finely tuned set of parameters?\n\n7. Potential for Temporal Artifacts/Bias: I would like to see discussion or analysis on whether incorporating prediction history could inadvertently introduce minor temporal correlations or specific texture biases into the generated images, even if the overall FID improves. A few qualitative examples showing cases where HiGS fails or introduces unintended artifacts (if any exist) would strengthen the paper.\n\n\n[1] Unified Continuous Generative Models. In page 7 of  https://arxiv.org/pdf/2505.07447, it proposes **Extrapolating the estimation**, which shares the same idea as this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ek5wsH7oW9", "reviewer": "Reviewer_2itd", "summary": "The author introduces a novel momentum-based sampling technique called History-Guided Sampling (HiGS), which improves the quality and efficiency of diffusion sampling by incorporating recent model predictions into each inference step. This approach yields more realistic outputs with finer details. Experiments across various diffusion models and datasets demonstrate promising improvements.", "questions": "* What motivated the choice of Equation 5 as the formulation for HiGS over other possible alternatives?\n\n* Could you elaborate on why \"the benefits of HiGS were most evident during the early and middle sampling steps,\" as stated in line 226? What underlying factors contribute to this behavior?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* The motivation is compelling: the authors propose that the Euler sampler used in diffusion models can be interpreted as performing stochastic gradient descent (SGD) on a time-varying energy function. Based on this perspective, they argue that the gradient estimate can be enhanced to improve both the efficiency and quality of the sampling process, incorporating a momentum-based term inspired by STORM.\n\n* HiGS is conceptually straightforward and can be easily integrated into pre-trained diffusion models. Moreover, it introduces minimal additional computational overhead.\n\n* The experiments are comprehensive in terms of datasets and model types, and the performance is persuasive.", "weaknesses": "* In Section 4.2, the methodology is presented in a relatively loose and unstructured manner. It would be beneficial to include clearer explanations of the design choices and a more thorough ablation study to identify which components contribute most to performance. Additionally, clarifying the rationale behind how these techniques are organized and integrated would strengthen the overall presentation.\n\n* The tuning required for the combination of techniques in the HiGS design may pose a practical burden for users, potentially limiting its accessibility and ease of adoption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)