Entropy Rectifying Guidance for Diffusion and Flow Models
Authors: Tariq Berrada
Adriana Romero-Soriano
Michal Drozdzal
Jakob Verbeek
Karteek Alahari
Venue: NeurIPS 2025 poster ()
Keywords: Generative model
diffusion
flows
guidance
sampling
image generation.
Primary area: deep_learning
TL;DR: We propose a novel guidance mechanism that significantly improves image quality, consistency and diversity.

Abstract: Guidance techniques are commonly used in diffusion and flow models to improve image quality and input consistency for conditional generative tasks such as class-conditional and text-to-image generation.
In particular, classifier-free guidance (CFG) is the most widely adopted guidance technique.
It results, however, in trade-offs across quality, diversity and consistency: improving some at the expense of others.
While recent work has shown that it is possible to disentangle these factors to some extent, such methods come with an overhead of requiring an additional (weaker) model, or require more forward passes per sampling step. 
In this paper, we propose Entropy Rectifying Guidance (ERG), a simple and effective guidance method based on inference-time changes in the attention mechanism of state-of-the-art diffusion transformer architectures, which allows for simultaneous improvements over image quality, diversity and prompt consistency. 
ERG is more general than CFG and similar guidance techniques, as it extends to unconditional sampling. 
We show that ERG results in significant improvements in various generation tasks such as text-to-image, class-conditional and unconditional image generation.
We also show that ERG can be seamlessly combined with other recent guidance methods such as CADS and APG, further improving  generations.

Entropy Rectifying Guidance for Diffusion and Flow Models
Tariq Berrada Ifriqi Adriana Romero-Soriano ,3,4,5 13.44
Michal Drozdzal'
Jakob Verbeek
Karteek Alahari?
FAIR at Meta 2 Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, IJK, France
McGill University
4 Mila, Quebec AI institute Canada CIFAR AI chair
tarigborradabmetta.com
Abstract
Guidance techniques are commonly used in diffusion and flow models to improve image quality and input consistency for conditional generative tasks such as class- conditional and text-to-image generation. particular, classifier-free guidance (CFG) is the most widely adopted guidance technique. It results, however, in trade-off's across quality, diversity and consistency: improving some at the expense of others. While recent work has shown that it is possible to disentangle these factors to some extent, such methods come with an overhead of requiring an additional (weaker) model, require more forward passes per sampling step. In this paper, we propose Entropy Rectifying Guidance (ERG). a simple and effective guidance method based on inference-time changes in the attention mechanism of state-of-the-arr diffusion transformer architectures, which allows for simultaneous improvements over image quality, diversity and prompt consistency. ERG is more general than CFG and similar guidance techniques, as extends to unconditional sampling. We show that ERG results in significant improvements in various tasks, including text-to-image, class-conditional and unconditional image generation. We also show that ERG can be seamlessly combined with other recent guidance methods such as CADS and APG, further improving generation results.
Introduction
Diffusion (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021; Dhariwal and Nichol, 2021) and flow models (Lipman et al., 2023; Ma al., 2024; Esser et al., 2024) are state-of-the-ar generative modeling tools for various modalities, ranging from images (Rombach et al., 2022; Podell et al., 2024; Chen et al., 2024; Esser et al., 2024), to audio (Wang et al., 2023; Levy et al., 2023), and video (Jin et al., 2025; Polyak et al., 2024). These models generate data by starting with simple prior and iteratively removing noise a process called "denoising". See Kingma and Gao (2023); Lipman al. (2023) for details diffusion and flow matching. These models can be conditioned inputs to control the generative process, e.g., in text-to-image models. Guidance techniques such as classifier guidance (Dhariwal and Nichol, 2021) and classifier-fres guidance (Ho and Salimans, 2021) are commonly used to improve sample quality and consistency with input conditioning. In the sampling process, these techniques combine conditional signal with unconditional one, and control the strength of conditioning through a scaling parameter. Although such guidance techniques are a crucial component in achieving state-of-the-ar results, they are also known to negatively affect the diversity of generated samples given a particular prompt (Sadat et al., 2024; Karras et al., 2024; Kynkäänniemi et al., 2024; Saharia et al., 2022). Moreover, too high guidance scales may lead to overly saturated images, affecting the quality of generated images (Saharia et al., 2022). An extensive analysis on the trade-offs of image generation quality, diversity, and consistency was presented by Astolfi et al. (2024). To mitigate these quality-diversiie conssstt--osssstneey trade-offs, offf, more advanced
various
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

CPG
ERG (ours)
CPG
ERG (ours)
Figure 1: Qualitative comparison of classifier-free guidance (CFG) and our Entropy Rectifying Guidance (ERG). The images generated using and diversity than standard CFG. Images are generated using 50 Euler steps; each column corresponds different random seed for the generations.
ERG (second and fourth row) exhibit greater quality
guidance techniques have recently been proposed, see e.g. Sadat et al. (2025); Karras et al. (2024). Most guidance techniques, however, require spending part of the training cycles on unconditional generation for the guidance to work, even if unconditional generation is not goal in itself, and are not applicable to unconditional sampling (Sadat al., 2025; Ho and Salimans, 2021). Others rely on a second model with weaker performance than the main model, thereby increasing memory requirements (Karras et al., 2024).
In our work, we build upon the work of Karras et al. (2024) and Hong (2024), and propose Entropy Rectifying Guidance (ERG): simple and effective method obtain both a strong and a weak predictive signal from a single model that leverages attention layers, where the model may be conditional unconditional. In particular, our method uses the Hopfield energy formulation of attention (Ramsauer et al., 2021; Hong, 2024) and applies a temperature scaling to the softmax function of the attention layers order obtain the weak predictive signal. This scaling does not require any adaptations in model training, and may be applied to pre-trained denoising models, as well as their accompanying text encoders. Moreover, motivated by this energy interpretation of attention layers, we also consider iterative re-application of attention layers, and rescaling the residual attention update. Our experiments show that manipulating the attention layers in this manner results in simultaneous improvements in sample quality, diversity, and consistency, contrary to the trade-offs observed by Astolfi al. (2024) for classifier-fe free guidance.
In summary, our contributions are the following:
1. We propose Entropy Rectifying Guidance (ERG), guidance mechanism based on modifying the energy landscape of the attention layers.
Since our guidance mechanism does not require unconditional inference, it directly applicable to any attention based diffusion or flow model, including unconditional, class-conditional, and text-to-image models.
3. Experimentally, we find that ERG significantly improves image quality and diversity while retaining the same prompt consistency as the standard classifier-free guidance (4-30 points in density, +4 points in VQAScore) on COCO at 512 resolution using our 1.9B text-to-image model.

---

2 Related work and background
In this section, we briefly review background material on diffusion and flow models, and related work on guidance techniques improve sampling well the Hopfield energy formulation of attention.
2.1 Diffusion and flow matching models
Diffusion models (Sohl-Dicksteinn et al., 2015; Ho et al., 2020; Song et al., 2021) form a flexible class of generative models whose underlying principle is to map samples from trivial unit Gaussian prior Po = (N I) to samples from a learned model p1 of the data distribution. The forward process is defined as: X2 0x + drE with E [0, 1], where X1 ~ Pl: and 04 is decreasing function of "time" 1 while o₂ an increasing function of t.
Flow matching methods (Lipman et al., 2023) assume that og 01 = 1 and 01 = -o = 0. Using these assumptions, during the reverse process x, interpolates between at f and x at = In contrast, score-based diffusion models (Ho et al., 2020; Dhariwal and Nichol, 2021) set and oe implicitly through different formulations of stochastic differential equations (SDE) where (0, I) is the equilibrium distribution. Additionally, they consider t [0. 2'] with T large enough so that Xy is approximately distributed as a unit Gaussian random variable.
2.2 Guidance mechanisms
Classifier guidance. To enable high quality conditional generation, Dhariwal and Nichol (2021) proposed to guide the sampling process by leveraging gradients from pre-trained auxiliary classifier p(c(x) in each denoising step. They use the classifier to define the (scaled) joint score function as Vx, log p(x,, c) x, log p(x)) + Vv. log p(c[x:), where p(xx) is an unconditional data model, and w is a scalar parameter regulating the strength of the classifier guidance. While classifier guidance allows to improve input consistency and image quality (at the expense of diversity), it requires an auxiliary classification model that is robust to ipputs X, with varying amounts of noise.
Classifier-free guidance. To avoid the need for an auxiliary noise-robust classifier, Ho and Sali- mans (2021) proposed classifier-free guidance (CFG). this case, during the training process, two generative models are learned, one conditional p(x|c) and one unconditional p(x(0) In practice, the unconditional model is trained by dropping conditioning information with a small probability. The score function used for sampling is extrapolated towards the conditional prediction and away from the unconditional prediction CCFG log p(x(c) = w V log p(x|c) + (1 w) w)Vx log p(x While CFG improves image quality and input consistency with respect classifier guidance, it tends to come at the cost of a reduction in diversity (Astolfi et al., 2024; Ho and Salimans, 2021; Sadat et al., 2024). Moreover, CFG often leads to generation artifacts, such as over-saturation as the guidance scale grows (Sadat et al., 2025).
Advanced guidance techniques. Several improved variants of classifier-free guidance have been pro- posed recently. Hong al. (2023) presented Self-Attention Guidance (SAG), a guidance mechanism based on feeding a modified intermediate sample x, when performing inference for unconditional prediction. The modification consists in blurring X; regions that are most attended to by the model's 'ss self-attention. This method has been developed for the U-Net architecture, hence applying it to more recent diffusion transformer architectures requires hyperparameter search to understand which attention layers should be used for this method.
Smoothed energy guidance (SEG) (Hong, 2024) contrasts conditional prediction with a "weaker" conditional prediction obtained by altering the attention's 's softmax energy with a Gaussian kernel applied to queries. This method developed for U-Net-style architectures and the softmax alteration applies to the self-attention layers in the middle block of the U-Net. In the conditional case. SEG uses a linear combination of the conditional, unconditional, and energy- smotthed unconditional prediction, whereas in our approach we only use the conditional and smoothed conditional term. Thus, SEG requires an additional function evaluation with respect to ERG for conditional inference. For unconditional inference, both approaches require only two function evaluations. In similar spirit, Ahn et al. (2024) propose a guidance method based on manipulating the attention mechanism, by replacing the attention matrix with identity mapping inside the denoiser U-Net.
3

---

Karras et al. (2024) proposed AutoGuidance, method that uses smaller/weaker version of the same conditional model for classifier-free guidance, resulting in better diversity and image quality. Like ours, their approach can also be applied to unconditional sampling. However, their method requires access to an earlier checkpoint of the model or, for best results, training a separate model with lower capacity, as well as accessing two models when sampling, which may increase the memory footprint.
Rather than considering modifications of the unconditional model term, Sadat al. (2024) proposed the irondition-annealee diffusion sampler" (CADS) to increase the diversity of generations while maintaining sample quality. This is achieved by adding Gaussian noise the conditioning tokens during inference, using (2025) propose "adaptive projected guidance" (APG), a variant of CFG that resolves the over- saturation problem by emphasizing guidance orthogonal to the conditional prediction, rescaling the guidance term, and introducing a negative momentum term. The latter two approaches can be easily combined with our approach, and we consider such combinations in our experiments.
seecewise-lineat decreasing schedule on the noise amplitude. Sadat et al.
Lastly, several works (Kynkäänniemi et al., 2024; Chung et al., 2024a; Pavasovic et al., 2025; Wang et al., 2024) explore non-constant weight schedules for CFG. Such methods are complementary to our work, as our method only operates at the architecture level by rectifying the attention updates. Although they therefore could be combined with our approach, we defer this to future work.
2.3 Hopfield energy formulation of attention
The Hopfield network (Hopfield, 1982) is dense associative memory model that aims to associate an input with its most similar pattern. More specifically, it constructs an energy function to model an energy landscape that contains basins of attraction around the desired patterns. Modern Hopfield energy networks (Ramsauer et al., 2021) introduce a new family of energy functions that improve the storage capacity of the model and make it compatible with continuous embeddings. Specifically, the following energy functional matches a continuous d-dimensional state (query) pattern & E IRd with N stored (key) patterns X (xi, xx) RaxN E(g: X) = 5 E LogSumEXxp (X Tx,,, 8),
where LogSumExp(xx 3) = 3-1 log (Eiz exp(a:)) where x, are the elements of the vector x. and [ scalar hyperparameter defining the sharpness of the approximation of the maximum in the LogSumExp operation. Intuitively, the first term imposes a finite norm on the queries while the second term measures the alignment between the state patterns (queries) and stored patterns (keys). Using the Concave-Convex Procedure (CCCP) (Yuille and Rangarajan, 2003), an iterative update rule, which converges to the global minima of the energy, can be derived as f X SoftMax(/XX T where SoftMax(x) = exp(x - LogSumExp(xx, 1)). Equivalently, each iterative update step can be seen as a gradient update in the negative direction of the energy Ve (E; X) = E X SoftMax (3X ()) Taking gradient descent step this energy landscape with step size results in an update of the form Ei+1 = E1 - 7 (Ei- XSX E.)). and for y = we recover the CCCP update.
Ramsauer et al. (2021) show that the CCCP update is related to the standard attention opera- tion as follows. Assuming there are S state (query) patterns, and N stored (key) patterns that can be mapped to keys, queries and values using linear transformations, he state pattern can be obtained through a concatenation: E = .......Es) E R Then the attention map is given by E XSofttMxxX E.). The keys, queries and values are obtained via linear pro- jections as K = XW Nxd, V = XW. Nxd and Q E44 WT Rdxd, re- spectively. By setting 3 1 and substituting this into the CCCP update rule, we obtain:
Va
V (W Hence, we observe the equa- tion of attention up to a linear transformation, which provides an interpretation of the attention mechanism through the Hopfield energy lens. Such a formulation has been used in the Energy Based Cross-Attention method (EBCA) (Park et al., 2024) for adaptive context control in order to incorporate additional contexts into the generative process of conditional diffusion model.
Q++1- SoftMax QKT K SoftMax QKT
Va
Va
3 Guidance via entropy rectification
Similarly to previous approaches (Ho and Salimans, 2021; Hong. 2024; Karras et al., 2024), our approach is based on contrasting the (conditional) denoising estimate with a less powerful one. In particular, using the Hopfield energy interpretation of attention (Ramsauer et al., 2021), see

---

Section 2.3, we manipulate the energy landscape of the attention layer. by rectifying the entropy of the associations in the attention operation. The modified attention layers lead to lower quality predictions, which are used as the contrasting term for guidance. This approach does not require second model, and can be used for both conditional and unconditional sampling. We refer to our approach as Entropy Rectifying Guidance (ERG).
3.1 Manipulating the energy landscape
Our method manipulates the energy landscapes by introducing two new test-time hyperparameters, a and T in the energy function:
E(::X) 1 -- -a a LogSSuEEEx (x T. B))
(I)
where B = vdd is the default temperature of the at- tention attention update. The temperature rescaling parameter 7 controls the sharpness of the softmax attention, and the relative importance of the simi- larity between the state matching term compared to the norm of the state patterns. Temperature rescal- ing with is similar to the Gaussian blurring of the attention maps introduced in SEG (Hong, 2024), but allows for non-local smoothing of the attention maps. Additionally, the view of the attention layer as a CCCP update of the energy function, allows for consideration of different methods to minimize the energy landscape. For instance, taking K gradi- ent descent steps with step size 7,, as illustrated in Algorithm 1.
Algorithm 1 Entropy rectifying guidance
Require: K N number of gradient updale sleps.
Require: > step size.
Require:
State pattern matching weight.
Require: > attention temperature.
Require: K, E RuX N keys and valnes.
Require: Q E 1x queries.
t-
while k: < K do
Q - Q
(Q sollmax (T. (T.9QKT) )v)
k
end while
Using different settings of the hyperparameters a, 7. T and K allows to manipulate the attention operation, and obtain noise estimates that deviate from the trained model. We expect these to be weaker estimates compared those provided by the model, as it was trained with standard attention layers, i.e. with y model, we refer to the method as image- ERRG, or I-ERG for short. In particular, we apply the negative//ncconditoona prediction part of the classifier guidance, and only on certain layers of the network that we will identify our experiments. Additionally, we impose kickoff threshold on the time steps in which guidance is applied.
K = 1. When applying this rectification mechanism to the denoiser
3.2 Manipulating the energy of the text encoder
Besides the image denoising model, we can also manipulate the energy landscape of -utention-based text encoders to obtain weak version of the conditional embeddings. Let c be the text prompt that used as conditioning. The text tokens are obtained by feeding the prompt to the text encoder: Ene(c) E i such as Llama (Grattafiori et al., 2024) or T5 (Chung et al., 2024b). Text tokens are then fed to the denoiser model through cross-attention layers.
To obtain a contrasting signal for guidance, we manipulate the energy landscape in the self-attention layers of the text encoder following Algorithm More specifically, for every self-attention layer in the text encoder, we introduce a temperature hyperparameter in the softmax function. This enables us change the strength by which keys and queries are being matched, resulting embedding at the output. For the remainder of the manuscript, we refer to this method condition- or C-ERG for short. For simplicity reasons, for the text encoder, we only consider changing the temperature but not the step size y, pattern matching weight a, and number of update steps K.
modified prompt
3.3 Guidance update
When combining the energy modulations in the text-encoder and denoising model, obtain Entropy Rectifying Guidance (ERG) update:
AERG t; 0g) w D(x, + (1 w) Df( o5. θg),
(2)
5

---

where D is the learned denoiser model with parameters θ that are omitted for simplicity, DE is the denoiser model where the attention layers have been replaced by the modified version presented in Equation (1), and fg {a. 7; T. K) the set of hyperparameters introduced by ERG. We use Φ. to denote the prompt embedding produced by the text encoder, while &² denotes the embedding obtained with the modified attention layers.
Compared standard CFG, the main differences are that (i) we replace the unconditional text embeddings with conditional embeddings obtained with the entropy attention mechanism (C-ERG), and (ii) the denoiser model for the negative/umconditiona predictions also uses modified attention layers (I-ERG). Finally, the changes to the image denoiser are only applied after a certain point during sampling in order not overly penalize the negative components of the ERG update at the start of sampling. For the text-encoder we apply the temperature scaling throughout the denoising process, so that at any stage we obtain a noise prediction that can be contrasted with the vanilla denoising signal.
Note that our approach can be combined with other approaches, e.g., CADS (Sadat et al., 2024) and APG (Sadat et al., 2025). We will explore such combinations in our experiments.
4 Experimental evaluation
4.1 Experimental setup
Datasets and architectures. We experiment with class-conditions and text-to-image models trained using rectified fow-matching (Lipman et al., 2023; Esser et al., 2024). We use face-blurred version of ImageNet (Deng et al., 2009) to train class-conditional models at 256 and 512 resolution based on the XL/2 variant of the DiT' architecture (Peebles and Xie, 2023), which is composed of 28 attention blocks with hidden dimension of 1152, resulting in 790M parameters. For the text-to- text-to-image model, use architecture similar to MMDiT (Esser al., 2024), and train 512 resolution model mix of a proprietary dataset of 320M text-image pairs and YFCCI00M (Thomee et al., 2016), where all faces in YFCCI00M have been blurred. Similar to MMDIT (Esser al., 2024), the model uses mix of different text encoders: Llama3-8B (Grattafiori et al., 2024) and F5---T5-XL (Chung et al., 2024b). During training, each of the text encoders is disabled with a probability of vO.T, so that the probability of both encoders being disabled is around 10%. We enable both text encoders during inference time for text-to-image generation, and disable both text-encoders for unconditional image generation experiments. The architecture of the model is made of 38 blocks with hidden dimension of 1, 536, resulting in approx 1.9B parameters. For both datasets, we recaption the images using both Florence-2 Large (Xiao et al., 2024) obtain medium-length captions and PaliGemma-3B (Beyer et al., 2024) for shorter COCO-style captions. Additional techniques to improve training efficiency of this model, such as conditioning mechanisms and pre-training strategies, were adopted from Berrada et al. (2024). For the class-conditional model we use the asymmetric autoencoder of Zhu et al. (2023), while for the larger eex-too-image model we use the SD3 autoencoder (Esser et al., 2024).
Metrics. We consider metries for quality, diversity, and consistency (Astolfi et al., 2024). We measure sample quality with FID (Heusel et al., 2017) and density (Naeem et al., 2020); sample diversity is measured with coverage (Naeem et al., 2020) and FID; and prompt consistency is measured with CLIPScore (Hessel et al., 2021) and VQAScore (Lin et al., 2024). For evaluation of text-to-image -mage and unconditional generation, we use the 40k COCO' 14 validation image-caption pairs. For the class-conditional models, we sample 50 images for each of the 1,000 ImageNet classes and the ImageNet validation set as a reference. All evaluated models are sampled using the Fuler method with 50 sampling steps. We use the EvaIGIM (Hall et al., 2024) library for all evaluations.
Baselines. In addition the standard classifier-free guidance, we compare our method to several recent state-of-the-an art guidance techniques: Condition-Anneet
Diffusion Sampler (CADS) (Sadat et al., 2024), Adaptive Projected Guidance (APG) (Sadat et al., 2025), Smooth Energy Guidance (SEG) (Hong, 2024), and Muito-Geidance (Karras et al., 2024). For APG, we follow the recommen- dations from the paper and set 7PPG = -0.5, TaPP = 0.0, TAPG = 5.0. For CADS, we perform a grid search over CADS E [0.6, 0.8). rz CADS E [0.8, 1.0), , CADS E [0.25. 1.0), s/CADS 1.0. Since SAG, PAG and SEG were developed specifically for the U-Net architecture, we adapt these method for diffusion transformers (Peebles and Xie, 2023) by applying the method in the attention layers

---

Table I: Comparison of ERG with other guidance approaches for text-to-image generation. We compare ERG to other state-of-the-ane art guidance approaches and mark the best result in each column in bold in the top part of the table. In the bottom part of the table evaluate combinations of ERG with APG and CADS, and bold results when they surpass the results in the upper part of the table.
Metric
FID (4)
Density (f)
Coverage (t)
CLIPScore (f)
VQAScore (t)
NFE (L)
Guidance
CRG (Ilo and Salimans, 2021)
12.81
98.24
71.12
26.45
70.15
2
APG (Sadat et al., 2025)
11.88
104.07
73.06
26.54
72.47
CADS (Sadat
2024)
101.01
72.99
26.76
73.36
PAG (Ahn et al., 2024)
12.75
107 21
72.20
26.80
73.32
2
SAG" Hong al. (2023)
11.68
103.58
72.74
26.81
72.16
2
SEG (Hong, 2024)
16.87
87.77
61.91
26. 86
T3.59
3
AutoGuidance (Karras 2024)
62.75
26.59
73.53
ERG (ours)
13.62
120.25
73.21
26.86
73.96
2
ERG (ours) APG
11.37
115.08
80.50
26.74
73.55
2
ERG (ours) + CADS
12.87
128.54
76.23
26.75
73.45
Cooittenry (t)
Consttterrg (e)
1.30
Qualty (1)
1.20
0.72
0.72
1.00
0.750
0.795
0.750
Quality (1)
Diomnit (1)
Divenity (1)
Mayg 5
Meau - 10
Macic 13
Menc 16
Figure 2: Pareto fronts on conissency-ddivesitt-quuiltt for text-to-image generation. Comparing ERG APG (dots) with APG (crosses). We sweep over different guidance scales (each marked with a different color), and hyper-parameters a, 7; T for ERG. Dashed lines trace the Pareto fronts for each plot. We measure consistency with VQAScore, quality with density and diversity with coverage.
of the middle blocks of the transformer; we refer to these methods with an asterisk superscript. Additional details on the choice of layers are provided in Appendix D.4. For AutoGuidance, we follow the recommendations from the paper and use an earlier checkpoint of the same model, at approximately 1 1/16-th 16th of the training, as the weaker model. To ensure a fair comparison, we select the best performing guidance strength for baselines as well as our method using the rank-scoring algorithm detailed in Appendix A.2. Note that this is different from reporting the optimal score achieved for each metric, which might not correspond to any particular run because of inherent trade-offs between the diffferent facets of the generations.
4.2 Main experimental results
Throughout our experiments, we modify attention layers in both the text encoder (C-ERG) and the image denoising model (I-ERG), unless specified otherwise.
Text-fo-image generation. In Table 1, we compare ERG with recent state-of-the-art guidance mechanisms on the text-to-image generation task. ERG demonstrates excellent performance, outper- forming baselines such as CFG, SAG, PAG and SEG" in most metrics. Specifically, ERG considerably boosts image quality as reported for Density (c.g.. +22 points when compared to CFG). Additionally, ERG achieves the highest consistency scores: +0.4 points in CLIPScore and +3.8 in VQAScore when compared to standard CFG. Moreover, ERG combined with APG achieves the overall best diversity, measured by FID and Coverage, and also improves Density over prior methods. These results suggest that our ERG approach is successfully able to boost the generation quality of the model in all three facets of generation (quality, diversity, and consistency). Moreover, the number of function evaluations (NFE) required for our approach is only two per inference step, which is comparable other methods except SEG which requires three.
Following Astolfi al. (2024), we plot Pareto fronts for quality (measured by Density), diversity (Coverage), and consistency (VQAScore) metrics in Figure 2 when using ERG APG. We find all
7

---

No Guidance
ERG (owrs)
Figure 3: Unconditional generation results. Compared to not using guidance (top), our ERG generates more realistic and detailed images and more coherent structure (bottom). Images obtained from T2I model at 512 with empty prompt as input. Samples in each column use the same seed.
Table Unconditional generation. Comparing Table Class-conditional generation. Com-
ERG to other approaches compatible with uncon- parison of ERG with other guidance methods for models trained for 256 and 512 resolution.
ditional sampling.
FID
Density (1)
Coverage
Res.
Density (1)
Coverae (1)
guidence
101.50
3,63
CFG dilo und Salimans, 2021)
3:67
127.00
85.81
(Tong
(Aha 2114)
PAG (Ali et 2024)
41.50
45.63
30.51
(Hang
SEG (Hong 2024)
31.75
55.56
$1.79
SEG' (Hong, 2024)
6.15
132 22
84.51
AutoCCuidanee (Kanas et al., 2024)
30.50
48.25
34.71
ERG (ours)
3.67
141 96
88.72
ERG
36.25
CFG (llo and Salimans, 2021)
5.63
146.97
86.70
PAG (Ahn al.. 2024)
6.65
134.
50.50
SAG (Hlng 2023)
612
6.81
120.00
83.01
(Hong, 2024) (ouzs)
the points belonging the Pareto fronts correspond ERG + APG, which improves in all three facets of the generations w.r.t. APG, and provides significant boosts for quality and consistency. This can also be observed in the qualitative comparison between APG and ERG APG in Figure 13 and Figure 14 in the supplementary material. Similarly, in Figure the images sampled using ERG show better visual quality than those sampled with CFG.
Unconditional generation. In the unconditional model sampling experiment, we compare I-ERG with sampling without guidance and using applicable methods: AutoGuidance, SEG", PAG and SAG'. We provide quantitative evaluation results in Table 2, where we find that ERG outperforms all other methods, with significant boosts in FID, Density, and Coverage over other methods. The qualitative examples in Figure clearly exhibit artifacts in terms structural coherence in all the objects present in the generated images when not using guidance, which disappear when using ERG.
Class-conditional generation. For class conisssoodiii generation, observe similar trends those seen for text-to-image and unconditional sampling in Table 3. In particular, at 256 resolution, we find improvements in Density and Coverage. FID remains similar CFG but is better compared to all other methods tested. At 512 resolution, ERG is best across all metrics, except for coverage where ERG is slightly behind CFG and SEG".
4.3 Analysis and ablations
Text attention energy. To isolate the effect of the temperature re-scaling in the text-encoder and image denoising components, we experiment with C-ERG and disable temperature rescaling in the image denoising model. In Figure 4, we vary the SoftMax temperature т, used in the text encoder, and consider generation performance for different guidance strengths w. For all metrics we find a somewhat symmetrical behavior around т, 1, which corresponds to an unguided prediction because in this case there is difference with the normal conditional prediction.

---

CLIPScore
Coverage
Density
FID
0.90
12
12.6
10.0
25
7.5
24
5.01
10
10
10
10
10
10
10
10a
10 10 10'
102
10
16'
10
2'-33
57
Figure 4: Temperature rescaling in the conditioning for text-to-image generation (C-ERG), We vary the text encoder's attention temperatures Tc- Each curve corresponds to different guidance strength w. The left-most point on each curve represents the result for standard CFG.
Table 4: Impact of the different components of ERG. Table 5: Multi-step gradient descent We accumulate different components of ERG, namely C-
for optimizing the energy landscape.
ERG, then I-ERG through denoiser entropy rectification
we ablate different values for K and %
and objective reweighting, finally all are merged into ERG.
10
1.5
0.5
0.1
FID
Densily
Coverage (1)
CLIP (1)
VQA (1)
FID [4)
13.62
13.62
13.06
12.68
K
K
X
12.81
98.24
71.12
26.45
70.15
Density
120.25
123.65
121.95
119.07
K
13.06
109.52
72.06
26.73
73.10
Coverage (1)
73.21
74.07
72.63
72.11
13.62
120.25
73.21
26.86
73.96
CLIP (1)
26.86
26.81
26.16
26.77
13.62
123.65
74.07
26.81
74.67
VQA (1)
73.96
74.67
73.01
73.95
The CLIPScore, Coverage and Density metrics generally improve when moving away from T. making it either larger or smaller. For FID, the best values are obtained with low guidance scales and intermediate temperature values, the general trend shows improved FID using C-ERG compared to standard guidance. Compared with standard lassifier-free guidance (left-hand side of the dashed lines), we find that all metrics can be improved for any guidance scale, provided with the right temperature. While To < results in higher CLIPScore, T, results in higher Coverage, indicating that tuning T. provides an easy way to control the diversity-consistency tradeoff.
Combining the different parts. Table 4, we combine different parts ERG and measure the effects different facets of image generation. Our results show that all components show positive effects across all metrics, at the expense a slight degradation in FID. Compared to the CFG baseline (first row), most of the improvement in CLIPScore and VQA brought by the conditional entropy rectification (T.. second row), while the improvements in Coverage and Density mostly come from rectifying the attention in the denoiser obtain (r, third row). Finally, a further modest improvement in Density, Coverage and VQA is brought by the update step size (7, fourth row).
Multi-step gradient descent. In Table we consider the effect of varying number of updates K for each attention operation along with the update step size. We find small variations in metrics with respect to the baseline K = gradient descent steps did not induce significant gains. Therefore, we used K = y = our default setup in our experiments, unless specified otherwise.
with slight improvements when setting 1.5, using multiple
5 Conclusion
We presented Entropy Rectifying Guidance (ERG), novel guidance mechanism for sampling diffusion and flow models which significantly improves sample quality without sacrificing diversity and consistency performance. In particular, by manipulating the energy landscape of the attention layers in the diffusion transformer and the text encoder inference time, ERG significantly boosts the performance of different models when studying their conitrteecsiiiiiiiiiiiii applicable to different modalities such as text-to-image, class conditional and unconditional models. Furthermore, ERG outperforms recent state-of-the-an guidance methods such as CADS, APG, SEG and AutoGuidance, while requiring the same compute as standard CFG. ERG can be combined with approaches such APG and CADS, which further improves results.
trade-offs and

---

Acknowledgements. Karteck Alahari was supported in part by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean Government (MSIT) (No. RS-2024--00457882, National Al Research Lab Project).
References
Donghoon Ahn, Hyoungwon Cho, Jaewon Min, Wooseok Jang. Jungwoo Kim, SeonHwa Kim, Hyun Hee Park. Kyong Hwan Jin, and Seungryong Kim. Self-rectifying diffusion sampling with perturbed-attention guidance. In European Conference on Computer Vision, 2024.
Pietro Astolfi, Marlene Careil, Melissa Hall, Oscar Mañas, Matthew Muckley, Jakob Verbeek, Adriana Romero Soriano, and Michal Drozdzal. conditional image generative models. arXiv preprint, 2406.10429 2024.
Pareto fronts of
Tariq Berrada, Pietro Astolfi, Melissa Hall, Reyhane Askari Hemmat, Yohann Benchetrit, Marton Havasi, Matthew J. Muckley, Karteek Alahari, Adriana Comero-Soriano, Jakob Verbeek, and Michal Drozdzal. On improved conditioning mechanisms and pre-training strategies for diffusion models. In Advances in Neural Information Processing Systems, 2024.
Lucas Beyer, Andreas Steiner, André Susano Pinto. Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Gryener, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bošnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, Ioana Bica, Ivana Balazevic, Joan Puigcerver, Pinelopi Papalampidi, Olivier Henaff, Xi Xiong, Radu Soricut, Jeremiah Harmsen, and Xiaohua Zhai. PaliGemma: A versatile 3B VLM for transfer. arXiv preprint, 2407.07726, 2024.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang. James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Fast training of diffusion transformer for photorealistic text-to- text-to-image synthesis. In International Conference on Learning Representations, 2024.
Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. CFG++: Manifold-constaule classifier free guidance for diffusion models. arXiv preprint, 2406.08070, 2024a.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. Scaling cnrruction-fintunnne language models. Journal of Machine Learning Research, (25), 2024b.
Deng. W. Dong, R. Socher, L.-J. Li, Li, and Fei-Fei. ImageNet: A large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition, 2009.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann. Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow trans- formers for high-resolution image synthesis. In International Conference on Machine Learning. 2024.
Aaron Grattafiori et al. The Llama herd of models. arXiv preprint, 2407.21783, 2024.
Melissa Hall, Oscar Mañas, Reyhane Askari, Mark Ibrahim, Candace Ross, Pietro Astolfi, Tariq Berrada Ifriqi, Marton Havasi, Yohann Benchetrit, Karen Ullrich, Carolina Braga, Abhishek Charnalia, Maeve Ryan, Mike Rabbat, Michal Drozdzal, Jakob Verbeek, and Adriana Romero Soriano. Evalgim: library for evaluating generative image models, arXiv preprint, 2412, 106044, 2024.
10

---

Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. In Conference on Empirical Methods in Natural Language Processing, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurlPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, In Advances in Neural Information Processing Systems, 2020.
Susung Hong. Smoothed energy guidance: Guiding diffusion models with reduced energy curvature of attention. In Advances in Neural Information Processing Systems, 2024.
Susung Hong, Gyuseong Lee, Wooseok Jang, and Seungryong Kim. Improving sample quality of diffusion models using self-attention guidance. In International Conference on Computer Vision, 2023.
J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8):2554-2555) 1982.
Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song. Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In International Conference on Learning Representations, 2025.
Tero Karras, Mïika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems, 2022.
Tero Karras, Miik Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with bad version of itself. In Advances in Neural Information Processing Systems, 2024.
Diederik P. Kingma and Ruiqi Gao. Understanding diffusion objectives as the ELBO with simple data augmentation. Advances in Neural Information Processing Systems, 2023.
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. In Advances in Neural Information Processing Systems, 2024.
Mark Levy, Bruno Di Giorgi, Floris Weers, Angelos Katharopoulos, and Tom Nickson. Controllable music production with diffusion models and guidance gradients. In Advances in Neural Information Processing Systems, 2023.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. arXiv preprint, 2404.01291, 2024.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. International Conference on Machine Learning, 2023.
Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Fijnden, and Saining Xie. SiT: Exploring flow and diffusion-based based generative models with scalable interpolant transformers. In European Conference on Computer Vision, 2024.
Muhammad Ferjad Nacem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jacjun Yoo. Reliable fidelity and diversity metries for generative models. In International Conference on Machine Learning, 2020.
Geon Yeong Park, Jeongsol Kim, Beomsu Kim, Sang Wan Lee, and Jong Chul Ye. Energy-based based cross attention for Bayesian context update in text-to-image diffusion models. In Advances in Neural Information Processing Systems, 2024.

---

Krunoslav Lehman Pavasovic. Jakob Verbeek, Giulio Biroli, and Mare Mezard. Understanding classifier-free guidance: High-dimensional theory and non-linear generalizations. arXiv preprint. 2502.07849, 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In International Conference on Computer Vision, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In International Conference on Learning Representations, 2024.
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching- Cao Chuang. David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang. Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Luxin Zhang. Mannat Singh, Mary Williamson, Matt Le, Matthew Yu, Mitesh Kumar Singh, Peizhao Zhang, Peter Vajda, Quentin Duval, Rohit Girdhar, Roshan Sumbaly, Sai Saketh Ramb- hatla, Sam Tsai, Samanch Azadi, Samyak Datta, Sanyuan Chen, Sean Bell, Sharadh Ramaswamy, Shelly Sheymin. Siddharth Bhattacharya, Simran Motwani, Tao Xu, Tianhe Li, Tingbo Hou, Wei- Ning Hsu, Xi Yin. Xiaoliang Dai, Yaniv Taigman, Yaqiao Luo, Yen-Cheng Liu, Yi-Chiao Wu, Yue Zhao, Yuval Kirstain, Zecheng He, Zijian He, Albert Pumarola, Ali Thabet, Artsiom Sanakoyeu, Arun Mallya, Baishan Guo, Boris Araya, Breena Kerr, Carleigh Wood, Ce Liu, Cen Peng, Dimitry Vengertsev, Edgar Schonfeld, Elliot Blanchard, Felix Juefei-Xa Xu, Fraylie Nord, Jeff Liang, John Hoffman, Jonas Kohler, Kaolin Fire, Karthik Sivakumar, Lawrence Chen, Licheng Yu, Luya Gao, Markos Georgopoulos, Rashel Moritz, Sara K. Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du. Movie gen: A cast of media foundation models. arXiv preprint, 2410.13720. 2024.
Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Holzleitner, Milena Pavlovic, Geir Kjetil Sandve, Victor Greiff, David Kreil, Michael Kopp. Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all you need. In International Conference on Learning Representations, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition, 2022.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M. Weber. CADS: Unleashing the diversity of diffusion models through condition-anneulle sampling. In International Conference on Learning Representations, 2024.
Seyedmorteza Sadat, Otmar Hilliges, and Romann M. Weber. Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In International Conference on Learning Representations, 2025.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to -ot-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems, 2022.
Jascha Sohl- Dickkstiin, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference Machine Learning, 2015.
Yang Song, Jascha -och--Dickttei,, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. YFCC100M: the new data in multimedia research. Communications of the ACM, 59(2):4473, 2016.
Xi Wang, Nicolas Dufour, Nefeli Andreou, Maule-Paule Cani, Victoria Fernandez Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. Tronsactions on Machine Learning Research, 2024.
12

---

Yuancheng Wang, Zeqian Ju, Xu Tan, Lei He, Zhizheng Wu, Jiang Bian, and sheng zhao. AUDIT: Audio editing by following instructions with latent diffusion models. In Advances in Neural Information Processing Systems, 2023.
Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing unified representation for a variety of vision tasks. In Conference Computer Vision and Pattern Recognition, 2024.
A. Yuille and Anand Rangarajan. The concave-convex procedure. Neural Computation, 15:915-925 2003.
Zixin Zhu, Xuelu Feng, Dongdong Chen, Jianmin Bao, Le Wang, Yinpeng Chen, Lu Yuan, and Gang Hua. Designing a better asymmetric VQGAN for StableDiffusion. arXiv preprint, 2306. 3306.0462. 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: we claim: (i) novel guidance technique to sample diffusion/flow models, (ii) our guidance technique is applicable to unconditional sampling, and (ii) improved sample qualiiyliversitylconssenyyy compared to previous methods. The claims are clearly stated in the abstract and introduction. Novelty is backup-up by a clear discussion of relevant related work, and performance claims are backed up by extensive experimental results on multiple data sets and models.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes].
Justification:
In section B of the appendix, we discuss the limitations of our method in detail.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
Justification: Our paper does not introduce theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our experiments are accompanied with the full set of hyperparameters used to reproduce each experiment, Namely Section 4.1 and Appendix E discuss the choice of hyperparameters in detail. We also provide pseudo-code for our method in Algorithm and Algorithm 2.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
15

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our paper provides pseudo-code and hyperparameter choices for popular open-source models such as Stable Diffusion 3, we evaluate our method on COCO dataset which open-source as well. Some models were trained on public data (ImageNet) others mix of public and proprietary data (YFCC 320M proprietary text-image pairs).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ca/
cublic/guides wwt.e//Saiissioooooooooooooooeee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ wtwpsscccppbllc/////cccc hwapollicgguides..eeeee /et////ooeeSeeeaeeeeeeeeee sisiisssion iesionnPoicyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Section 4.1 provides comprehensive context for the setup in which exper- iments were conducted. Our main results Section 4.2 are accompanied with extensive ablations in Section 4.3 where each experiment is accompanied with a thorough discussion the setup of the experiment and an analysis of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
16

---

Justification: While we do not provide statistical significance tests, we evaluate our method in different settings (class-conditional, text-to-image). different models (internal models of different scales, open-source models), and also models trained for different resolutions (256 and 512), across all these setups, we find our method to result significant improvements. Accordingly, we believe that the significance of our method is validated by the extent of our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our method consists in a modified sampling algorithm for diffusion and flow models, the new sampling method has comparable NFEs to standard classifier-free guidance, which we report in Table 1. The main difference in compute compared would be induced by the number of gradient descent steps K in the multistep energy update, this variable is set to K = in most of our experiments and recommended setups, hence the computational cost is practically unchanged compared to standard classifier free guidance.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: httpps:////uriige
Answer: [Yes]
Justification: Our internal model was trained on mitigated datasets (CSAM filtering + face blurring), only used compliant datasets for evaluation (MS COCO), which is
17

---

under Creative Commons Attribution 4.0 License. Our implementation on top of open- source models does not alter the model weights. but just provides changes to the guidance mechanism used for inference.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: We propose a method to improve the quality of generative models, hence our method can be used to generate high-quality, realistic content which can be used maliciously spread misinformation and disinformation, creating deepfakes, identity theft etc. By releasing our code transparently, we provide way for researchers to study and counter the potential harmful effects of our method being misused, allowing for the development of defense strategies.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly. harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
We do not release any new models or datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
18

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We carefully cite the respective authors of each method/model and dataset that was referenced in our manuseript. We additionally provide links to the different assets used in Table 14.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi snwerswwthcodeee thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA].
Justification:
We do not introduce new datasets/coddemoodels with our paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: We do not perform any crowdsourcing or use human subjects in our work.
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not perform any crowdsourcing human subjects in work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: Our work does not make use of LLMs an important, original or non-stande otanstandard component of the method we propose.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMss as any important, original, non-standard components.
Please refer to LLM policy (https: : /ttp:///ee..../eeeeee cc/ C for what should or should not be described.
20


Reviews:
[{"review_id": "X24ZpIQiyY", "reviewer": "Reviewer_QhcC", "summary": "This paper presents Entropy Rectifying Guidance (ERG), a guidance mechanism for diffusion and flow models designed to simultaneously improve the quality, diversity, and input consistency of generated samples, including text-to-image, class-conditional, and unconditional generation tasks. ERG operates by manipulating the attention mechanism, specifically using a Hopfield energy-inspired temperature scaling and attention modification at inference time. This yields a weaker predictive signal contrasted with the original model, thereby providing guidance without the need for an additional model or costly model evaluations. Empirical results demonstrate that ERG outperforms standard Classifier-Free Guidance (CFG) and several recent advanced baselines.", "questions": "Please refer to Weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. ERG does not require model retraining, works with pre-trained transformer-based diffusion/flow models, and applies to text-to-image, class-conditional, and unconditional setups.\n2. The approach extends the energy landscape of attention to invent a new, simple guidance mechanism that is broader than prior methods relying on auxiliary or weaker models.\n3. This paper is well-written and well-organized. The proposed method, ERG, is easy to follow.\n\nWeaknesses:\n1. Although ERG, compared to CFG, can be applied to unconditional image generation, the performance improvement, especially in terms of FID scores, is relatively minimal.\n2. The evaluation metrics used in this paper are not very standard. Commonly used metrics, such as Inception Score (IS), sFID, Precision, and Recall, are not reported, which somewhat diminishes the persuasiveness of the experimental results.\n3. Other guidance techniques, such as CFG, can be applied to autoregressive image generation models. Could ERG similarly improve the performance of such models as well?", "was_revised": true, "final_justification": "All my concerns have been addressed.", "rebuttal": "We thank the reviewer for their thoughtful and encouraging feedback. We are particularily grateful that they appreciate the clarity and organization of the paper, the versatility and simplicity of ERG, and its ability to enhance diverse generative tasks without the need for retraining or auxiliary models.\n\nBelow we address the questions raised by the review.\n\n**W1.** “*Performance improvement is relatively minimal, especially in terms of FID*”\n\nWe acknowledge the reviewer’s observation. Our evaluation setup is designed to optimize **overall generation quality**, balancing *fidelity, diversity, and consistency*—not just a single metric like FID. This makes direct comparisons on FID alone potentially misleading.\n\nAs shown by [3], FID-optimal checkpoints often lie far from the Pareto front when jointly considering realism, consistency, and diversity, and in models such as SDXL, optimizing for FID can significantly reduce realism (Section 4.1, Fig. 3). \nDhariwal et al. [4] further show that lower guidance scales minimize FID and recall, while higher guidance scales improve perceptual sharpness and structure, measured by IS and precision even though they increase FID (Figure 3). \n\nHowever, to directly address this concern, we conducted an additional experiment where we **tuned each method specifically to optimize FID**, selecting the best hyperparameters (e.g., guidance scale, ERG scale) for each.\n\n| Method | CFG | ERG | APG | ERG+APG | PAG | SAG | SEG | CADS | ERG+CADS | AG |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| FID | 5.25 | **4.93** | 6.62 | 5.24 | 7.00 | 5.28 | 6.72 | 5.12 | 5.00 | 7.11 |\n\nCFG achieves its best FID at a guidance scale of **1.25**, while ERG achieves its optimal FID of **4.93** using a guidance scale of **1.25** and an ERG scale of **2.0**. This demonstrates that ERG is capable of outperforming prior guidance methods (e.g., SEG, APG) when optimized for FID alone, in addition to providing better trade-offs under broader evaluation criteria as can be seen in the Pareto fronts in Figure 2 of the main paper.\n\n**W2.** “*The evaluation metrics used in this paper are not very standard. Commonly used metrics, such as Inception Score (IS), sFID, Precision, and Recall, are not reported, which somewhat diminishes the persuasiveness of the experimental results.*”\n\nWe thank the reviewer for their insight on this point. We originally chose to report a subset of metrics that robustly capture the distinct facets of generative performance without overwhelming the reader. Specifically, we believe that density and coverage (D\\&C) offer more reliable and interpretable fidelity and diversity evaluations than precision and recall (P\\&R), as detailed in \\[1\\]. While P\\&R have advanced the separation of fidelity and diversity, they exhibit key limitations: \n\n* They fail to detect matches between identical distributions, often reporting suboptimal values even when real and generated data are sampled from the same distribution (\\[1\\] \\- Fig. 3).\n* They are vulnerable to outliers, as a single real or fake outlier can inflate the manifold estimation and artificially boost scores (\\[1\\] \\- Fig. 3, left).\n* They depend on arbitrarily chosen hyperparameters, such as the number of nearest neighbors k, without principled guidance for selection ([1] - sec. 3.1).  \n* In contrast, D\\&C address these issues by using a superposition-based manifold estimation that is more robust to data sparsity and outliers, yield analytically tractable behavior under ideal conditions ([1] - sec. 3.3), and support systematic hyperparameter tuning ([1] - sec. 3.4). \n\nTherefore, we opted for D\\&C as they faithfully reflect the fidelity-diversity trade-offs of generative models in both synthetic and real-world scenarios.\n\nThat said, we fully agree on the value of standard benchmarks for comparability. We now include Inception Score (IS), sFID, and Precision & Recall for completeness.\n\n| Method | CFG | APG | CADS | PAG | SAG | SEG | AutoGuidance | ERG | ERG+APG | ERG+CADS |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| Precision | 65.71 | 66.34 | 66.42 | 66.42 | 64.97 | 61.22 | 61.48 | 70.92 | 69.50 | **72.72** |\n| Recall | 43.95 | 45.15 | 45.75 | 43.94 | 47.98 | 36.88 | 34.60 | 41.43 | **50.25** | 38.89 |\n| sFID | 26.53 | 20.31 | 18.20 | 27.21 | 28.55 | 33.80 | 14.75 | 19.56 | **9.30** | 15.32 |\n| IS | 37.25 | 43.76 | 40.75 | 40.50 | 38.28 | 40.64 | 41.81 | 43.16 | **53.47** | 42.52 |\n\nFor precision & recall, we observe that similar trends to density and coverage with ERG+CADS achieving the best precision (72.72). ERG on its own achieves a precision of 70.92 compared to 65.71 for CFG.\nRecall is slightly lower when using ERG alone (41.43 vs 43.95 for CFG), but ERG+APG achieves the best recall of 50.25.\nFor Inception Score, ERG improves over the CFG baseline (43.16 vs 37.25 for CFG) and achieves the best score when combined with APG (53.47).\nSimilarly, ERG+APG results in significant sFID and Inception score improvement, with ERG alone on par with APG.\n\n**W3.** “*Other guidance techniques, such as CFG, can be applied to autoregressive image generation models. Could ERG similarly improve the performance of such models as well?*”\n\nWe appreciate the reviewer's suggestion. While guidance methods like CFG have been successfully applied to autoregressive image generation, extending ERG to this setting is non-trivial and would require further research. \nIn diffusion and flow-based models, such guidance can be interpreted theoretically due to the iterative nature of the sampling process which operates on the global image representation.\nIn contrast, models like LlaMAgen \\[2\\] sequentially predict the series of $h \\times w$ tokens without access to the full image context, no refinement process similar to diffusion/flow sampling occurs.\nWe conducted a preliminary investigation using the LLaMAgen model but did not observe stable improvements with an out-of-the-box implementation of ERG compared to CFG.\n\nThis suggests that directly transferring ERG to autoregressive architectures may not be straightforward, potentially due to differences in the way attention and generation unfold across time steps in such models, which renders some parameters in ERG incompatible, such as the I-ERG kickoff threshold which does not have a direct parallel in the autoregressive setting.\n\nNonetheless, we believe this is a promising direction for future work and are interested in exploring how ERG-like mechanisms could be adapted to autoregressive settings more effectively.\n\n\\[1\\] Reliable fidelity and diversity metrics for generative models. Naeem et al., ICML, 2020.\n\n\\[2\\] Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation, Sun et al., 2024.\n\n\\[3\\] Consistency-diversity-realism Pareto fronts of conditional image generative models, Astolfi et al., NeurIPS, 2024.\n\n\\[4\\] Diffusion Models Beat GANs on Image Synthesis, Dhariwal et al., NeurIPS, 2021."}, {"review_id": "SzBwNQ0Lyz", "reviewer": "Reviewer_Rztq", "summary": "The paper presents Entropy Rectifying Guidance (ERG), an inference-time guidance method presented as an alternative to classifier-free guidance (CFG) for diffusion/flow-matching models. ERG operates by modifying the energy landscape of attention layers within a diffusion/flow model. Particularly, the softmax temperature in attention is modulated to produce a weaker conditional signal/prediction, which takes the place of the unconditional estimation in CFG. A particularly interesting outcome of ERG is that it can be used to improve unconditional sampling, unlike CFG. The authors provide a set of convincing qualitative and quantitative experiments to demonstrate that ERG improves quality, diversity, and consistency.", "questions": "Q1. In section 4.1, the authors mention that they train their own diffusion/flow models for each of the experiments. Can the existing method not be used as a drop-in replacement for CFG in pre-trained diffusion models?\n\n- If training is necessary: why is it required? It seems to me that ERG can be used in pre-trained models, unless I missed something.\n- If training is not necessary: why not provide some experiments with existing open-source pre-trained models?\n\nQ2. Instead of converting some prior work on attention-based UNet architectures to DiT for comparison (SAG, PAG, SEG, noted in L263), it would have been better if the authors could have directly applied their method on the attention-based UNets and compared against the previous methods without modifying them. It will address W1.1. [Unless, ERG requires training a model from scratch (as I noted in Q1), in which case this comparison is not necessary.]\n\nQ3. From my understanding, SEG uses an extra NFE simply because it wants to utilize the already available unconditional pretrained model. Can it not be used similar to ERG in the conditional case without an extra NFE?\n\n- The following ablation would help make things clear: (a) disable temperature scaling in the text encoder (i.e. only using I-ERG), to compare against SEG (w/o extra NFE, if possible). This would prove that the non-local smoothing in ERG (L181) is clearly better than the Gaussian blurring in SEG.\n\nQ4. In L34, the authors note that there is an oversaturation problem in CFG. How does standalone ERG tackle this problem? I did not see any experiments addressing this.\n\n### Suggestions\nS1. I believe there is a small mistake in L304, `At 512 resolution, ERG is best across all metrics, except for coverage where ERG is slightly behind CFG and SEG*`. Referring to Table 3, it seems ERG is better than SEG*, but worse than PAG*. It should probably be `...behind CFG and PAG*` in L304.\n\nIn general, I like the paper, but am concerned with some of the choices in the evaluation procedure (e.g. not using pre-trained model, converting attention UNet-based method to DiT for comparison, etc.).\n\nIf I misunderstood something in the paper or mentioned something that has already been done in the paper, please feel free to clarify my mistake.\n\nI am willing to raise my score by 1 point if some questions are addressed.", "limitations": "Yes, I believe the authors have addressed their limitations well.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\nS1. The paper is well written in general. The proposed method, ERG, is well-motivated, aiming to address some of the known shortcomings of CFG.\n\nS2. The proposed method is technically sound, built upon the idea of (a) using a weaker prediction for guidance, and (b) using the hopfield energy formulation of attention to create the weaker prediction.\n\nS3. The authors compare their approach with several baselines for text-to-image generation, class-conditional generation, and unconditional generation. According to the results, ERG seems to simultaneously improve quality, diversity, and prompt consistency, which highlights its potential usefulness.\n\n### Weaknesses\n\nW1. The proposed method seems to be relevant only for a particular family of model architecture (e.g. models with attention layers). Classifier-free guidance is a very general technique that is invariant to the architecture of the generative model.\n\n-  W1.1. Though ERG is theoretically applicable to any diffusion model with attention layers, the experiments seem to have only been performed on DiT-based architectures. It would be more convincing if the paper also covered some UNet architectures that have self-attention (e.g. EDM2).\n\nW2. Classifier-free guidance takes a geometric weighted average between conditional and unconditional probabilities, i.e. $\\hat{p}(x \\mid y) \\propto p(x \\mid y)^{\\omega}p(x)^{1 - \\omega}$. It has a good probabilistic interpretation based on the score function, and can be derived from the expansion of the score function from classifier guidance. It is a bit unclear as to how an unconditional model is being guided by using a weaker unconditional model, as shown in ERG, as both models are unconditional. Is there a probabilistic interpretation of this, beyond \"contrasting with weaker prediction\"?\n\nW3. It is not entirely clear what are the specific novel concepts introduced in this paper over the previous work Smoothed Energy Guidance (SEG). To my understanding, it seems that---(a) modifying the energy landscape of attention layers in diffusion models, and (b) using this phenomenon for performing a sort of guidance in conditional and even unconditional generation---have been previously established by SEG. The primary difference seems to lie in the details of how exactly the energy is modulated (temperature rescaling in ERG vs Gaussian blurring in SEG). Clearly highlighting the differences with SEG will elucidate the strengths of this paper.", "was_revised": true, "final_justification": "My strongest concern was that the experiments in the main paper were done on \"privately trained\" models, despite the availability of numerous open source diffusion/flow-matching models to report results on. The authors have addressed this in the rebuttal.\n\nAnother concern was that the method ERG lacked some theoretical foundation; in the main paper, the method is presented along the lines of \"we do this and it just works like CFG\". The authors provide a better perspective in the rebuttal, which makes the paper stronger.\n\nDue to these two main reasons (omitting smaller reasons for brevity), I am raising my score from Borderline Accept to Accept.", "rebuttal": "We thank the reviewer for their thoughtful feedback. We are also grateful for their acknowledgement of the motivation and technical soundness of ERG, the broad experimental work validating the method’s effectiveness across diverse settings and their positive comments on the clarity and organization of the paper.\n\nBelow we address the questions raised by the review.\n\n**W1.** \nYes, our method is based on hopfield energy interpretation of the attention so it is only applies to models with attention layers. We believe that this limitation is not major as the vast majority of large-scale state-of-the-art models currently adopt DiT-based architectures (see SD \\>= 3.0, FLUX, SORA, Bagel, WAN, MovieGen etc.).  \n\nCompared to some previous works, our method is less restrictive.\nFor instance, Autoguidance requires access to a weaker model, and SEG/PAG were only developed for UNet models with attention, their practical usefulness for large scale SOTA models (DiT) remained unclear.\n\n**W1.1.**\nThe reason for focusing on DiT architectures lies in their architecture inducing a more complex design space for methods such as SEG, and because they have become the standard for large-scale generation. However, we agree that providing evaluations on different architectures would strengthen our work. Following the setup in Section F of the appendix, we provide additional (I-)ERG results for SDXL, PixArt-alpha, DiT and EDM2 (autoguidance). SDXL and EDM2 use UNet architectures while PixArt-alpha uses local window attention. For SDXL and Pixart-alpha, we use the opensource version from Hugging Face alongside the EvalGIM evaluation framework. For EDM2 and DiT, we use the evaluation setups from their respective open-source implementations.\n\n| Model\\Metric | FID | CLIP | Density | Coverage | Precision | Recall |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| PixART-alpha | 28.64 | 26.03 | 52.98 | 13.46 | 47.85 | 43.38 |\n| PixART-alpha w. ERG | 27.75 | 26.61 | 70.61 | 16.57 | 56.40 | 42.86 |\n| SDXL | 21.65 | 29.54 | 80.93 | 20.00 | 60.68 | 49.17 |\n| SDXL w. ERG | 20.94 | 29.97 | 84.05 | 20.21 | 60.66 | 54.39 |\n\n| Model \\ Metric | FID | sFID | Precision | Recall | IS |\n| :---- | :---- | :---- | :---- | :---- | :---- |\n| DiT-XL/2-256 | 2.71 | 4.55 | 82.77 | 57.94 | 277.96 |\n| DiT-XL/2-256 \\+ ERG |  2.36 | 4.32 | 86.9 | 60.03 | 295.03 |  |\n\n| Model \\ Metric | FID | FID\\_dinov2 |\n| :---- | :---- | :---- |\n| EDM2-IN-XXL-512 | 2.00 | 59.34 |\n| EDM2-IN-XXL-512 \\+ ERG | 1.35 | **31.32** |\n| EDM2-IN-XXL-512-autog | **1.33** | 31.56 |\n\n**W2.**\nWe thank the reviewer for this interesting question. We provide two interpretations of ERG based on entropy regularized RL and variational inference.\n\nLet $p$ be the density function for the strong model and $p^\\tau$ the one induced by the attention rectification mechanism we introduce in the paper.\n\n## I-ERG\n\nWe begin with the I-ERG guidance update, which modifies the model’s score estimate as:\n\n$\n\\nabla_x \\log p^\\tau(x) = \\nabla_x \\log p(x) + w \\cdot \\left( \\nabla_x \\log p(x) - \\nabla_x \\log p^\\tau(x) \\right)\n$\n\nRewriting the right-hand side, we obtain:\n\n$\n\\nabla_x \\log p^\\tau(x) = \\nabla_x \\log p(x) + w \\cdot \\nabla_x \\log \\left( \\frac{p(x)}{p^\\tau(x)} \\right)\n$\n\nIntegrating both sides with respect to $x$, we find:\n\n$\np^\\tau(x) \\propto p(x) \\cdot \\exp \\left( w \\cdot R(x) \\right), \\quad \\text{where } R(x) = \\log \\left( \\frac{p(x)}{p^\\tau(x)} \\right)\n$\n\nThis form reveals that I-ERG guidance parallels a posterior distribution that reweights the base model’s density $p(x)$ by an exponential function of the reward signal $R(x)$.\n\nThis exponential reweighting aligns with objectives used in **entropy-regularized RL** or **KL control**, where the optimal distribution maximizes an expected reward under a KL penalty with respect to a prior:\n\n$\n\\pi^* = \\arg\\max_{\\pi}  \\mathbb{E}_{x \\sim \\pi}[R(x)] - \\frac{1}{\\lambda} \\mathrm{KL}(\\pi \\Vert  p)\n$\n\nHere, $\\lambda$ serves as an inverse temperature controlling the sharpness of reward influence. The solution to this optimization problem has the form:\n\n$\n\\pi(x) \\propto p(x) \\cdot \\exp(\\lambda R(x))\n$\n\nThis shows that the I-ERG update implicitly implements the maximum entropy policy framework, steering the sampling distribution toward high-reward regions while maintaining proximity to the base model $p(x)$. Equivalently, using Bayes rule, it can be viewed as defining a **joint distribution** over $x$ under $p(x)$ and the energy induced by reward $R(x)$, (assuming conditional independence).\n\n## C-ERG\n\nIn the conditional setting, assume we start from a model $p(x|s)$ conditioned on some semantic signal $s$. Let $s^\\tau = s + \\Delta s$ represent a degraded or weakened version of the conditioning (e.g., via temperature scaling or perturbation).\n\nThe C-ERG guidance update is given by:\n\n$\n\\nabla_x \\log p^{ERG}(x|s, s^\\tau) = \\nabla_x \\log p(x|s) + w \\cdot \\left( \\nabla_x \\log p(x|s) - \\nabla_x \\log p(x|s^\\tau) \\right)\n$\n\nOr, equivalently:\n\n$\n\\nabla_x \\log p^{ERG}(x|s, s^\\tau) = (1 + w) \\cdot \\nabla_x \\log p(x|s) - w \\cdot \\nabla_x \\log p(x|s^\\tau)\n$\n\nTo analyze this further, we perform a first-order Taylor expansion of the degraded score $\\nabla_x \\log p(x|s^\\tau)$ around $s$:\n\n$\n\\nabla_x \\log p(x|s^\\tau) \\approx \\nabla_x \\log p(x|s) + \\frac{\\partial^2 \\log p(x|s)}{\\partial x \\, \\partial s^\\top} \\cdot \\Delta s\n$\n\nSubstituting this into the update:\n\n$\n\\nabla_x \\log p^{ERG}(x|s, s^\\tau) \\approx \\nabla_x \\log p(x|s) - w \\cdot \\left( \\frac{\\partial^2 \\log p(x|s)}{\\partial x \\, \\partial s^\\top} \\cdot \\Delta s \\right)\n$\n\nThus, C-ERG introduces a **Jacobian-vector product correction**, nudging the sample along directions where the score is most sensitive to perturbations in the conditioning. This helps reinforce fine-grained semantic detail in generation, even when the conditioning signal is noisy or coarsened.\n\nThis Jacobian-vector product has a natural interpretation: it reflects how the image-level score $\\nabla_x \\log p(x|s)$ changes when the conditioning $s$ is slightly perturbed. This resembles **classifier guidance**, where one steers the generation based on $\\nabla_x \\log p(y|x)$ — the gradient of a classifier with respect to the image.\n\nHere, the conditioning is a continuous vector $s$, and C-ERG guides generation using:\n\n$\n\\frac{\\partial^2 \\log p(x|s)}{\\partial x \\, \\partial s^\\top} \\cdot \\Delta s\n$\n\nwhich is the **direction in image space most sensitive to changes in the semantics of $s$**. This amounts to amplifying features that are especially responsive to semantic precision.\n\n**W3.** The distinction between our method and Smoothed Energy Guidance (SEG) can be better clarified. While SEG also modifies attention energies in diffusion models for guidance—both in conditional and unconditional settings—the key differences lie in the mechanism of rectification and its broader applicability.\nBelow, we summarize the main novelties of our method over SEG:\n* ERG modifies attention energies not only in the denoiser but also in the text encoder, allowing finer control over consistency and diversity.\n* Our ablations demonstrate that temperature rescaling (ERG) consistently outperforms the Gaussian blurring approach used in SEG.\n* SEG shows an unfavorable tradeoff between quality and consistency when applied to DiT architectures. We address this by introducing a kickoff threshold, which controls steerability while preserving high quality.\n* ERG is compatible with DiT-based models (and UNet models with attention) we provide clear implementation guidelines for models of varying sizes. SEG paper provides a limited scope in comparison.\n* Unlike prior works that report only limited metrics such as FID and IS, we conduct extensive evaluations across multiple generation settings. We thoroughly examine trade-offs between quality, diversity, and consistency, and offer practical insights for applying ERG across architectures.\n\n**Q1&2.** Retraining models is not required for our method. We chose to use our internal model to adhere to internal guidelines and, more importantly, to ensure tighter control over key variables that often differ across open-source models—such as training data, architecture, image resolution, task type, and the specifics of the diffusion/flow scheduler and sampler. By using an internal model, we minimize confounding factors and enable a fairer comparison of guidance methods.\nTo address the reviewer’s concern, we have also added a new set of experiments using publicly available implementations of popular open-source models (see the tables above). Thus, we now cover SD 3.0, SD 3.5 (Table 8 of the appendix), SDXL, PixArt-alpha, DiT, and EDM2 (tables above).\n\n**Q3.** We believe the requested ablation is presented in Table 6 & Figure 17, and discussed in Section C.3 of the appendix. We compare I-ERG with a kickoff threshold $\\kappa = 0$ with SEG, hence the only difference under this setting is the attention rectification mechanism (Gaussian blurring vs entropy rectification). Results reported showcase better detail modeling qualitatively and better scores for FID, density, coverage and CLIPScore with entropy rectification than with Gaussian blurring. If the reviewer has more specific questions about these experiments, we would be happy to address them.\n\n**Q4.** We indirectly address this in section C.2 of the appendix.\nBy analyzing the orthogonal and parallel components of the guidance term, we find ERG to provide a stronger orthogonal signal wrt CFG, enabling better quality at lower guidance strengths, which reduces oversaturation issues.\nWe will better clarify this link in the paper.\n\n**S1.** Thank you for pointing out this typo. It will be corrected in future versions of the paper.\n\n[1] Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control, Domingo-Enrich et al., ICLR 2025."}, {"review_id": "x2wb0kL3u5", "reviewer": "Reviewer_1okZ", "summary": "This paper proposes a novel guidance method to improve sampling quality by updating the model via test-time adaptation. Specifically, during the sampling process, the authors reinterpret the attention mechanism in the self-attention layer as a Hopfield energy formulation and perform updates that descend the Hopfield energy landscape. This approach achieves state-of-the-art results across various evaluation metrics, including FID, recall, coverage, density, prompt fidelity, and VQA scores.", "questions": "Please see weaknesses.", "limitations": "Please see weaknesses.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n- The idea of updating attention layers at test time, inspired by hopfield energy is novel.\n- The method is simple yet effective.\n- The paper is well-written.\n- The authors conduct thorough comparisons across many evaluation metrics (FID, recall, coverage, density, prompt fidelity, VQA score).\n- Extensive experiments are conducted on various models and datasets.\n- The method also performs well in unconditional sampling scenarios.\n\n# Weaknesses & Questions\n- In Figure 4, the hyperparameter τ, which is not derived from the Hopfield energy function, appears to be very influential. It would be beneficial to provide a theoretical explanation or motivation for its inclusion.\n- Although the method outperforms others on most metrics, it slightly underperforms in FID compared to some baselines. Is there a justification or insight into why this might be the case?\n\n# Minor Weaknesses\n- Line 151: Typo. “he state patter” → should be corrected.", "was_revised": true, "final_justification": "The rebuttal provided clear and convincing explanations for my concerns about the hyperparameter τ and the slight FID underperformance. Given these resolutions and the paper’s overall strength, I am raising my recommendation from Borderline Accept to Accept.", "rebuttal": "We thank the reviewer for their thoughtful and encouraging feedback. They highlighted the novelty of adapting attention layers at test time using a Hopfield energy perspective, as well as the simplicity and effectiveness of the proposed method. We appreciate their recognition of the thoroughness of our experimental evaluation across a wide range of metrics, models, and datasets. Their acknowledgment of the method's strong performance in both conditional and unconditional sampling scenarios is also greatly valued.\n\nBelow we address all the questions raised by the reviewer.\n\n**W1. Motivation for $\\tau$** We thank the reviewer for highlighting the importance of the temperature hyperparameter $\\tau$. While not part of the original Hopfield energy formulation, τ is a theoretically grounded extension that modulates the sharpness of the softmax in the *LogSumExp* term of the energy function (eq. (1) in the paper). Specifically, it enables control over the entropy of the attention distribution, allowing us to interpolate between diffuse and sharply peaked attention. \n\nMathematically, introducing $\\tau$ rescales the similarity scores as $\\log \\sum_i \\exp(\\tau \\beta x_i^T \\psi)$, corresponding to a softmax temperature adjustment. \n\nFrom a statistical mechanics perspective, $\\tau$ acts as an inverse temperature in a *Boltzmann distribution*, balancing exploration (low $\\tau$, high entropy) and exploitation (high $\\tau$, low entropy) when matching attention keys and queries. \n\nCrucially, introducing $\\tau$ does not break the concave-convex structure of the energy function, and the CCCP optimization remains valid and convergent. This allows us to embed $\\tau$ directly into the energy landscape and perform test-time entropy manipulation in a principled manner. \n\nUnlike prior methods such as SEG, which apply local Gaussian smoothing post hoc, our approach integrates $\\tau$ into the optimization dynamics via CCCP descent, enabling non-local and theoretically interpretable control over attention.\n\nWe will clarify this motivation and its theoretical basis in the revised paper.\n\n**W2. Under performance in terms of FID**\nWe thank the reviewer for pointing out the slightly higher FID values in some configurations. Our evaluation strategy is designed to prioritize balanced performance across fidelity, diversity, and consistency, rather than optimizing for FID alone. \n\nAs shown by [1], FID-optimal checkpoints often lie far from the Pareto front when jointly considering realism, consistency, and diversity, and in models such as SDXL, optimizing for FID can significantly reduce realism (Section 4.1, Fig. 3). \nDhariwal et al. [2] further show that lower guidance scales minimize FID and recall, while higher guidance scales improve perceptual sharpness and structure, measured by IS and precision even though they increase FID (Figure 3). \n\nIn line with these findings, we adopted a multi-objective evaluation strategy. To directly address this question, we conducted an additional experiment optimizing hyperparameters for FID. We will include results (presented below) for completeness.\n\n| Method | CFG | ERG | APG | ERG+APG | PAG | SAG | SEG | CADS | ERG+CADS | AG |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| FID | 5.25 | **4.93** | 6.62 | 5.24 | 7.00 | 5.28 | 6.72 | 5.12 | 5.00 | 7.11 |\n\n\n**Minor weaknesses** \nThanks for pointing out this typo, it will be corrected in future versions of the paper.\n\n[1] Consistency-diversity-realism Pareto fronts of conditional image generative models, Astolfi et al., NeurIPS'24.\n\n[2] Diffusion Models Beat {GAN}s on Image Synthesis, Dhariwal et al., NeurIPS'21."}, {"review_id": "tKuMpVJgRP", "reviewer": "Reviewer_R39s", "summary": "This paper proposes Entropy Rectifying Guidance (ERG) which acts as a inference time modification of attention. ERG claims to improve quality and diversity simultaneously. Furthermore the authors also show their method applied to the text encoder and diffusion models separately.", "questions": "- Why is ERG so much better in Coverage in the unconditional case while being only marginally better in FID and density?\n- Why is there such a sharp change in measured performance in Fig 8?", "limitations": "The authors discuss limitations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Weaknesses:\n- Not enough transparancy in terms of evaluation. Datasets should be stated more clearly.\n- The authors should include ImageNet 256 x 256 using a pretrained DiT-XL/2 and cite the original metrics from the respective papers. \n- Further, a comparison using sFID, InceptionScore, Recall and Precision would be good as this gives even more ground for fair comparisons. \n\nStrengths:\n- Overall well written and interesting concept.\n- Metrics support their general claims even though i would like to see a bit more common metrics as means of comparison (see W2/W3)\n- Very good performance in the unconditional generation which is interesting.", "was_revised": true, "final_justification": "After the rebuttal, the authors have answered all questions I had and explained any topic I was confused on in a precise manner. Therefore, i am raining clarity/quality to 3 and the final rating to Accept.", "rebuttal": "We thank the reviewer for their valuable feedback. \nWe are especially encouraged that they recognized the novelty of ERG as an inference-time attention modulation and appreciated its ability to improve both quality and diversity. \nWe are glad they found the concept interesting and the paper well written. \nTheir acknowledgment of ERG’s strong performance in the unconditional generation setting is very much appreciated. \n\nBelow we address all the questions raised by the reviewer.\n\n**W1.** “*Not enough transparency in terms of evaluation. Datasets should be stated more clearly.*”  \nWe thank the reviewer for raising this point. We discuss the datasets and the evaluation setup in Section 4.1.  \nWe will make sure to update the experimental section to separate dataset setup and architecture details in order to improve clarity.  \nIf the reviewer points to specific information that they think is missing from the text or not clearly explained, we would be happy to address it during the discussion period and update it in the paper.\n\n**W2.** \"*The authors should include ImageNet 256 x 256 using a pretrained DiT-XL/2 and cite the original metrics from the respective papers.*\"\nAs suggested, we evaluated with DiT-XL/2 using the open-source implementation, and show the results below. The positive influence of ERG is clearly demonstrated in these results.\n\n| Model \\ Metric | FID | sFID | Precision | Recall | IS |\n| :---- | :---- | :---- | :----: | :---- | :---- |\n| DiT-XL/2-256 | 2.71 | 4.55 | 82.77 | 57.94 | 277.96 |\n| DiT-XL/2-256 \\+ ERG |  2.36 | 4.32 | 86.90 | 60.03 | 295.03 |  |\n\n**W3.** \"*A comparison using sFID, InceptionScore, Recall and Precision would be good as this gives even more ground for fair comparisons.*\"\n\nWe thank the reviewer for this suggestion. The extended list of metrics for our main results in Table 1 of the paper is presented below.\n\n| Method | CFG | APG | CADS | PAG | SAG | SEG | AutoGuidance | ERG | ERG+APG | ERG+CADS |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :----: | :---- | ----: | :---- |\n| Precision | 65.71 | 66.34 | 66.42 | 66.42 | 64.97 | 61.22 | 61.48 | 70.92 | 69.50 | **72.72** |\n| Recall | 43.95 | 45.15 | 45.75 | 43.94 | 47.98 | 36.88 | 34.60 | 41.43 | **50.25** | 38.89 |\n| sFID | 26.53 | 20.31 | 18.20 | 27.21 | 28.55 | 33.80 | 14.75 | 19.56 | **9.30** | 15.32 |\n| IS | 37.25 | 43.76 | 40.75 | 40.50 | 38.28 | 40.64 | 41.81 | 43.16 | **53.47** | 42.52 |\n\nFor precision & recall, we observe similar trends to density and coverage with ERG+CADS achieving the best precision (72.72). ERG on its own achieves a precision of 70.92 compared to 65.71 for CFG.\nRecall is slightly lower when using ERG alone (41.43 vs 43.95 for CFG), but ERG+APG achieves the best recall of 50.25.\nFor Inception Score, ERG improves over the CFG baseline (43.16 vs 37.25 for CFG) and achieves the best score when combined with APG (53.47).\nSimilarly, ERG+APG results in significant sFID and Inception score improvement, with ERG alone on par with APG.\n\n**Q1.** “*Why is ERG so much better in Coverage in the unconditional case while being only marginally better in FID and density?*”\n\nERG shows a stronger improvement in coverage in the unconditional case, while gains in FID and Density are more modest. This is largely due to our multi-metric optimization strategy: we tune our models to perform well across all metrics—coverage, FID, and density—rather than optimizing exclusively for one.\n\nCoverage captures the diversity of generated samples, whereas density reflects sample quality, and FID balances both. The significant boost in coverage specifically stems from applying a scheduled attention temperature, which allows ERG to gradually increase the influence of guidance over time. In contrast, SEG applies strong guidance from the very beginning, which tends to over-optimize for quality early in generation at the expense of diversity and consistency. This tradeoff is discussed in Section C.5 and illustrated in Figure 15 of the appendix.\n\n**Q2.** “*Why is there such a sharp change in measured performance in Fig 8?*”\n\nAs discussed in Section C.4 of the appendix, we observe that attention layers exhibit distinct functional roles across the network, which can be broadly grouped into three stages: (1) encoding (layers 0–10, low entropy), (2) modeling (layers 10–25, high entropy), and (3) decoding (layers 25–38, low entropy). We find that modifying the rectification mechanism in either the encoding or decoding stages leads to sharp drops in performance due to the emergence of low-level artifacts in the generated outputs. This explains the abrupt change observed in Figure 8, and the same effect can be seen in Figure 16\\."}]

Author rebuttal:

Meta-review: This paper presents Entropy Rectifying Guidance (ERG), a guidance mechanism for diffusion and flow models designed to simultaneously improve the quality, diversity, and input consistency of generated samples, including text-to-image, class-conditional, and unconditional generation tasks. ERG operates by manipulating the attention mechanism, specifically using a Hopfield energy-inspired temperature scaling and attention modification at inference time. This yields a weaker predictive signal contrasted with the original model, thereby providing guidance without the need for an additional model or costly model evaluations. Empirical results demonstrate that ERG outperforms standard Classifier-Free Guidance (CFG) and several recent advanced baselines.

The reviewers liked the creativity, novelty, simplicity, and soundness of the method. They raised various concerns, which the authors addressed successfully during the discussion period. In the end, all reviewers endorsed the acceptance of the paper.
Decision: Accept (poster)