Rethinking Entropy in Test-Time Adaptation: The Missing Piece from Energy Duality
Authors: Mincheol Park
Heeji Won
Won Woo Ro
Suhyun Kim
Venue: NeurIPS 2025 spotlight ()
Keywords: Test-Time Adaptation
Domain Adaptation
Energy-Based Models
Primary area: optimization

Abstract: Test-time adaptation (TTA) aims to preserve model performance under distribution shifts. Yet, most existing methods rely on entropy minimization for confident predictions. This paper re-examines the sufficiency of entropy minimization by analyzing its dual relationship with energy. We view energy as a proxy for likelihood, where lower energy indicates higher observability under the learned distribution. We uncover that entropy and energy are tightly associated, controlled by the model’s confidence or ambiguity, and show that simultaneous reduction of both is essential. Importantly, we reveal that entropy minimization alone neither ensures energy reduction nor supports reliable likelihood estimation, and it requires explicit discriminative guidance to reach zero entropy. To combat these problems, we propose a twofold solution. First, we introduce a likelihood-based objective grounded in energy-based models, which reshape the energy landscape to favor test samples. For stable and scalable training, we adopt sliced score matching—a sampling-free, Hessian-insensitive approximation of Fisher divergence. Second, we enhance entropy minimization with a cross-entropy that treats the predicted class as a target to promote discriminability. By counterbalancing entropy and energy through the solution of multi-objective optimization, our unified TTA, ReTTA, outperforms existing entropy- or energy-based approaches across diverse distribution shifts.

Rethinking Entropy in Test-Time Adaptation: The Missing Piece from Energy Duality
Heeji Won Won Woo Ro Suhyun Kim Samsung Advanced Institute of Technology, 'Samsung Electronics, KKorea University. Yonsei University, *Kyung Hee University danime. park@samsung........ com, gmlx11026@korea. kc.kr, wro@yonsei. ac kr, dr. suhyun, kaiiuuniiimmmmmim com
Mincheol Park
Abstract
Test-time adaptation (TTA) aims to preserve model performance under distribution shifts. Yet, most existing methods rely on entropy minimization for confident predictions. This paper re-examines sufficiency of entropy minimization by an- alyzing its dual relationship with energy. We view energy proxy for likelihood, where lower energy indicates higher observability under the learned distribution. We uncover that entropy and energy tightly associated, controlled by the model's confidence or ambiguity, and show that simultancous reduction of both is essential. Importantly, reveal that entropy minimization alone neither ensures energy reduction nor supports reliable likelihood estimation, and it requires explicit dis- criminative guidance reach zero entropy. To combat these problems, propose a twofold solution. First, we introduce a likelihood- based objective grounded in energy-based models, which reshape the energy landscape favor test samples. For stable and scalable training, we adopt sliced score matching-4 sampling-free Hessian-insensitive approximation of Fisher divergence. Second, we enhance en- tropy minimization with a cross cross-entropy that treats the predicted class as a target to promote discriminability. By counterbalancing entropy and energy through the solution of multi-ojjective optimization, our unified TTA, ReTTA, outperforms existing entropy- or energy-based approaches across diverse distribution shifts.
Introduction
Deep learning models are increasingly ubiquitous in cutting-edge technologies such autonomous vehicles [21 17 and biomedical science 22 key assumption behind their success that test data comes from the same distribution training data. However, this assumption is often broken in practice. Test data can be subject to degeneration, referred to as covariate shifts, such as changes in lighting due to weather conditions unexpected noise caused by sensor degradation (14). Unfortunately, the meckiinn-making models degrades due to the distribution shifts [3]. This poses substantial challenge for the practical deployment of pre-trained models.
Test-time adaptation (TTA) has been proposed to yield more reliable decision-makee under distribu- tion shifts. An early TTA method centered on minimizing the entropy of test data (34), assuming models could separate classes well [1]] This key idea became the cornerstone of state-of-the-ar methods 19 20, 26 27 40), fueling the evolution of TTA. These modern approaches adopt proxy techniques in parallel, such as data filtering (26) or pseudo-labeling [TO 23 (35], because data streams experience dynamic shifts (38 and high correlation (non-i.i.d.) (2) in practice. Still, the core principle of minimizing entropy remains unchanged, even as many efforts are made to address the limitations of optimizing entropy alone.
*Co-corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this study, we raise a crucial question: Is minimizing entropy truly sufficient as the core objective for TIA? Our revisit to entropy through its dual, energy [8] which reflects the likelihood of being in-distribution, reveals key gap: the lack of momentum to minimize energy for better likelihood estimation in the test domain. More recently, an energy-based TTA method [39] aims to reduce energy overall, but it overlooks the advantages of minimizing entropy, which enhances discriminability.
This paper argues that the complete reduction of entropy and energy is crucial. However, even for data with low confidence, achieving zero entropy is difficult, even when entropy and energy are minimized simultaneously. Our test supports this by analyzing the distribution of entropy and energy in test data (Figure []. The distribution reflects whether the logit would be confident or ambiguous prediction, and implies discrete bands following a log-shaped curve (Figure [d)) Here, the lower energy and entropy correlate with correct predictions. However, this observation also reveals that, with only the two objectives, test data cannot transition between these bands to improve accuracy. This, in short, signifies the need for an additional objective to guide the model toward discriminability.
This paper introduces ReTTA, a novel unified TTA based on both entropy and energy with two objectives. The first objective is to utilize energy-based models (EBMs) [18]. where the marginal density of data is modeled by an energy function, typically defined as the LogSumExp of the model's logit. EBMs allow the model to reshape its likelihood in response to data 244. with training focusing on maximizing the density by lowering the energy of test data and raising the energy of generated samples [39]. However, sampling during EBM training can be unstable [5]. We approximate this
process using a first-order method. This reformulation allows for minimizing Fisher divergence between the test and model distributions by aligning their scores [31. This sampling-free objective well-suited for TTA. Precisely, to avoid unstable loss due to abnormal Hessian values, we adopt sliced score matching (SSM), which provides scalar comparative loss for TTA (3)
The second objective is to achieve complete entropy minimization (EM) by incorporating a dis- criminative objective that guides the model's prediction toward a single class. To this end, we use cross-entropy, targeting the most probable class, which we define as the targeted class convergence (TCC) loss. By combining EM loss with SSM and TCC, we form the unified ReTTA loss. Here, entropy and energy optimization should be handled carefully because the lack of supervision can hinder convergence, especially given the unpredictable nature of distribution shifts. To address the challenge of balancing entropy and energy optimization, we propose self-adjusting coefficient, where energy optimization is adaptively adjusted relative to EM, regardless of the type of distribution shift. Extensive experimental evaluations demonstrate that ReTTA outperforms TTA approaches that rely solely on entropy or energy optimization.
Our contributions are summarized follows:
We confirm that minimizing entropy alone is insufficient for estimating the test distribution's ss likelihood, emphasizing the necessity for simultaneous entropy and energy minimization in TTA. We address this with sampling-free energy adaptation loss (SSM), which, combined with EM, directly maximizes likelihood.
We establish that successful TTA requires energy reduction and convergence to the lowest entropy. We propose the targeted class convergence (TCC) loss, using cross-entropy, and integrate it with EM and SSM in novel unified EM objective, ReTTA
We propose a self-adjusting coefficient to counterbalance the optimization of entropy and en- ergy, effectively addressing challenges such as unpredictable distribution shifts. Evaluations demonstrate that ReTTA adaptively works on various corruption data and performs well.
2
Related Work
Test-time adaptation (TTA). TTA aims to improve model generalizability under distribution shifts. This is achieved by updating model parameters using test data, without access to the training dataset.
Numerous TTA techniques have been proposed, including pseudo-labeling 23 10 35), calibration of
normalization layers 8 28
38 27 consistency- regularization [30], prototype aligmment [5],
low-rank mixtures of experts 19, and energy adaptation [39] Among these, EM 34
26
27 20
remains a widely adopted objective, encouraging confident predictions during adaptation. However,
Boudiaf et al. (2 highlight the failure of EM when the test stream lacks diversity. More recently,
Choi et al. [3] point out the limitations of relying solely on EM in such unpredictable scenarios,

---

emphasizing the importance of energy. Building upon these insights, we offer a new perspective by showing why EM, while useful, is insufficient on its own. Additionally, we explore the eneroyyntroppy relationship and argue that, to improve the effectiveness of entropy minimization, it is essential to reduce energy and boost the model's discriminability concurrently.
Energy-based models (EBMs). EBMs [88] are class of nonmoormll probabilistic models with an intractable normalizing constant. They use stochastic approximations to estimate this constant, offering flexibility in parameterization and enabling the modeling of a wide range of distributions [12]. Through these approximations, EBMs generate data via energy functions, without relying on an explicit neural network. This flexibility has led to applications in tasks such as image generation [4] 6).
domain adaptation 41 [37], and domain generalization 36). Recently, energy-based methods for TTA have focused reducing energy within the model's distribution enhance generalizability (38). However, the method ignores the direction of energy alignment and requires multiple sampling iterations due to the intractable constant. While adaptive energy adaptation [3] attempts address these issues, it relies on heuristics and mini-batch configurations. In contrast, our work introduces more scalable approach through Sliced Score Matching (SSM) [33], providing sampling- free objective that improves TTA while avoiding the problems [5] in training EBMs.
3 Analytical Motivation and Observation in Entropy Minimization
Preliminaries. Let the source dataset D, be sampled from the training data distribution pa (x, v). and the target dataset D, from the test data distribution Pr(x, y). A discriminative model fo RD RK parameterized by 0, which maps data x RD to K real-valued outputs, is trained by maximizing the log- posstertoror log p( (D.) for the source dataset. During testing, the model fo infers the label y for unseen test (target) data x2 from the K classes by marginalizing over the parameters @ as:
plu xx; D.) = p(uvlx:: ())(())))))))))))))))))))))) (D)..)
(1)
where (xq: y1) De. Covariate shift occurs due to the shift in the marginal distribution of data, i.e. px(x) A px)).. When covariate shifts exist, the joint distribution also differs, i.e., L...,,((,,
Ps(Xx, y) (x, y),
which compromises inference in Eq. by causing a mismatch in likelihoods and leads to degraded accuracy. As workaround, many TTA methods [20 26 (7 34] attempt to update the parameters 0 applying EM. In Section 3.1 and 3.2
we discuss what is missing in the EM during TTA.
3.1 Rethinking Entropy Minimization
The mitigation of the covariate shift is key to the success of TTA. Specifically, given the model parameterized by 0, which estimates the source distribution p. (x, y), one promising approach is to update 0 to maximize the likelihood of Pt (x) when test data is observed, i.e., px(x) (x)...
In this regard, we account for p(x) as the sum of its factorized component y, i.e., p(x) Ey, p(x, y). However, test time, the label y is unknown. TTA approaches treat the probable classes as potential labels, assuming that the model, having once maximized the log- log-liielihodd on the source domain, has strong discriminative power [II] Therefore, the probable classes are determined by the model's output probabilities p. which is computed by applying the Softmax to the logits:
exp(So(x)[x))
p(x) = [p(y 0). .. = K 0]]; p(y|x; 0) = Σg exp(fo(x))
(2)
where p(x) e R and fx( represents the logit of the data Here, we note whether the likelihood of the data, marginalized over these probable classes, is maximized by minimizing the entropy of
the Softmax p. To this end, it is necessary introduce quantity that quantifies how the data likely
belongs to the marginal distribution parameterized by 0. We use the energy [8 39). Ea which maps data or its logit deterministic scalar by summing over the probable classes, defined by:
Ex(x) == - log Σ exp(fe(x)(k)).
(3)
This log partition function, also defined as Eg (z) = - LogSumExp(z) with z = fx)),, indicates that larger negative value represents more likely (or highly observable) data under the distribution px(x).
Now, we focus on the relationship between energy and entropy. We note that they form a conjugate pair, exhibiting duality that helps understand the trajectory of energy during minimizing entropy.
3

---

K=1000, 2-0.5
-16
Energy
Energy
Entropy
Entropy
Entropy
Entropy
No adaptation
(h) EM
(c) ReTTA (ours)
(d) Energy-entropy band
Distribution of test data based on energy and entropy values, and visualization of energy- entropy bands. (a) No adaptation: inference without TTA. (b) EM: results from the state-of-the-ar EM method [27]. (c) ReTTA (ours): unified TTA integrating EM with the two objectives from Section 4 (1,350 more correct samples than FM). To observe this phenomenon, ResNet-50 (BN) is applied to the contrast corruption (severity 5) from ImageNet-C [14], following [27]. (d) Energy- entropy band:
Figure 1:
energy-entropy curves with respect to the number of classes K and the secondary logit Z.
Lemma (Conjugate Relation). Suppose represents the model's logit, and g denotes the gradient of the concave function Eg with respect to the logit z. The concave conjugate of Es(z) is defined as E (g) = minz(g" z - Eg(z)). Then, the gradient g corresponds negatively to the Softmax, i.e., g = (z) = -xx), and the conjugate function Egg (g) becomes the negative entropy of p(x):
(g) H(p) -p()" log p(x).
Eq. holld in reverse when considering the conjugate of negative entropy (for clarity, we use "entropy" to reter to negative entropy). Thus, both functions exhibit bi-duality, with each being the conjugate of the other. Building on the bi-duality, we consider the following Fenchel duality.
exhibit bi-duality. The Theorem). Primal function Ee(z) and its conjugate function E3 (g) as Ez) = mingig"
Lemma 2 (Fenchel-Moreat
can be completely recovered from its conjugate function E (g) Eg (g)). Therefore, energy and entrupy satisfy the following relationship:
(z) mint min(-p"z Hp)).
(5)
Analytical motivation. The duality in Eq. 5 provides valuable insight. When entropy is minimized, (p) - O. the model's 's output p becomes a one-hot vector. Accordingly, the product p z approaches the logit of the most confident class k* i.e., Fz(z)) = which corresponds to the overall energy. In other words, minimizing entropy does not provide a clear momentum to reduce the overall energy. In short, EM updates model parameters increase confidence for the confident classes in test data, but lacks an objective to maximize the likelihood of the marginal distribution.
3.2 Observation for Energy and Entropy Relationship
In this section, we examine another problem of minimizing entropy alone. We confirm this by visually observing how the energy and entropy of test data change under the influence of EM. Figure [@a] shows that, when TTA is not applied, test data exhibit high entropy and energy, with the distribution concentrated around these high values. When EM works (Figure (bb), test data converge toward a region where entropy approaches zero, with energy moving toward larger negative values.
Intriguingly, the distribution of test data with respect energy and entropy suggests log-shapes relation. with their outermost curve acting as an upper bound. Here, we view this curve as new perspective for understanding the tight interaction between energy and entropy. Specifically, as we found, by applying restricting conditions to the model's 's logits, we can define a function capable of interpreting the distribution of energy and entropy.
Theorem 1. Suppose the logit of the model fo is defined over K classes, where k classes are assigned a primary logit z* with strong influence, and the remaining K k classes share a singular logit Z with minimal influence. Then, the closed-form equation for the energy-entropy relationship bused on the conditioned logits is given by:
H(E)) = -(1 Eo) log
50 Es),
(6)
k
4

---

where Eg E IR denotes the energy, and H E [0, log K] represents the entropy. (kk) is a variable defined by C(k) (K- k)e?.
From Figure [(d) Eq. 6 that the energy-entrops relationship forms "bands," which represent sets of closely spaced function values that the conditioned logits can occupy, depending the discrete value of the remaining classes k. These bands thus make it easy to infer the possible values of the logit for the test data. For instance, Figure [(b) shows that if multiple primary logits exist (i.e.. ho = 2, 3) with distinct values, the data can be distributed across the bands, or when the logits singular, the data will lie each band. Here, we note one phenomenon: test data near the zero-entropy and low-energy band (k = 1) appear to be corrected.
Motivation. Our motivation stems from the fact that minimizing entropy alone makes it difficult for the Softmax of the logits to converge to the zero-entropy region of the band, k = 1. This difficulty arises from the non-zero probabilities in the Softmax, which cause different convergences for the logits of each class [3] This issue becomes particularly apparent in TTA, where the model observes and updates the data once during inference [3). Therefore, an additional goal should be to guide the logits toward the k = 1 band explicitly, where entropy approaches zero.
Methodology
Building on both motivations, we present two TTA objectives in conjunction with EM: (1) maximizing the likelihood of marginal distribution, and (2) guiding the logits of x, toward zero-entropy region.
Energy-based models. We model the marginal distribution using EBMs 18] Specifically, we define the joint distribution over test data x; and a possible class Ye based on the model's logit as (o(x,, ye) = exp(fe (x. ( ly (()), where Z(0) denotes the normalizing constant [2]. Marginaliz- ing out the class variable yt, obtain the marginal distribution over x,;
po (x,) = exp( (x,))
Z(0)
where the energy Ex. follows Eq. The normalizing constant 2(0) = See, exp(-Eo(xx)) )xxx is
intractable, which poses a challenge in optimizing the log-likelihoode of (xx): max Ep, [log po (x:)] ]- We revisit this challenge in Section 4.1 and introduce a proxy objective for stable parameter updates.
4.1 Sliced Score Matching Loss
The derivative of the expected log-density Ep. [log (xx))) encourages the model to decrease the energy of the test data while increasing the energy of confabulations (samples generated by the model). Formally, the derivative is given by:
(8) The first expectation term Ep..1] in Eq. 8 involves generating samples (confabulations) from the model distribution (x+). typically achieved through a Markov Chain Monte Carlo (MCMC) method, e.g., Gibbs sampling. Among various MCMC techniques. Langevin dynamics is widely adopted as
EpE,, [log po (x(x)) [x)] Epg [x [ Ex
representative gradient-based sampling approach [5 16 39]:
= -
Vo, + μue,
6 6~ C~,(O,ID).
where the Markov chain is initialized from the test data, i.e., x. X2- Here, l controls the step size, and is Gaussian noise added at each iteration. However, Eq. optimization of Eq. 8 [. and it typically requires repeated iterations. To combat the instability and inefficiency associated with MCMC-based sampling, we adopt an alternative (32]. We leverage the fact that a single-step Langevin update, applied to data sampled from the true distribution Pt (x,), provides a good approximation to the gradient of the Fisher divergence between (x(x)) and (xx).
9 aloone is often insufficient for stable
Lemma one-step Langevin update initialized from x, px(x;) approximates the gradient of the Fisher divergence between the true distribution (x(x)) and the model distribution px(x;) parameterized by 6, as follows:
V VoEp, (x.)) 112
VoDr Vo + o(12),
(10)
where Dr (plll) is the Fisher divergence, and o( denotes higher-order term with respect to μ².
5

---

This formulation justifies the use of the Fisher divergence as a surrogate objective for likelihood- basedd training in EBMs when employing a one-step Langevin update. In particular, the Fisher divergence is also known as score matching, and is defined as:
D Dr(Di(x))llpoo
pp. [I log (x,,)(
Xx, log log
(11)
2 VVx, log ~ Ep. [T log)o(x;))
(12)
where Vx log p(x) is the score function, which characterizes how the log-density of p(x) varies with respect to x. Unlike Eq. this score matching Eq. "q..Tennabll enables sampling-free optimization via simple Monte Carlo estimator based solely on empirical averages over the test data. However, computing score matching requires evaluating the trace of the Hessian V2, of the model's s log-density in Eq. 12 This term is costly to compute [25] and can be overly sensitive to the sharp local curvature. Therefore, the use of score matching may become limited in high-dimensional data.
In this paper, we leverage Sliced Score Matching (SSM) 33 a variant of score matching that scales well to high-dimensional data. The key idea is to match inner products of score functions along randomly sampled directions, instead of matching the full score values directly:
D&F (Pe (x)))x)))))x))) Bp. Xx, (xi) - V'V. log po (x, )l|",
(13)
2 (x.)
|| V log (x:)
(14)
where (sF(p) is the sliced Fisher divergence and p(v) is chosen normal distribution. (0, ID).
Other valid choices for p(v) should satisfy Eppv) [vv ] 0 and Ep(v)]llv
op [33] Building
on this formulation, we construct the following unbiased estimator as a proxy for maximizing the
log-density of px(x.). defined by Eq. 7
0 Ex(xi) dxy day
un + Σ (
D
(2E((x))
)
ESSM (0) = B
Σ X.e
(15)
2
Oak
1=1
where Br represents a mini-bated of test data xs, sampled by D. Eq. that concentrates enhancing the TTA of EM.
defines our first objective
4.2 Targeted Class Convergence Loss
One key challenge remains: when Softmax in Eq. 2 is applied to the logits with respect to the data Xr, resulting in non non-confident predictions (i.e., the model does not favor a single class, such as the zone of k: 2,3, ... in Eq. 6). EM alone for prediction pxx,) is insufficient achieve zero-entropy convergence. This is especially challenging in the context of TTA, where the model observes the data once and updates once during inference
In other words, full convergence requires a well-defined target and appropriate supervision. To this end, leveraging the model's 's discriminative power [II], we treat the most probable class from the Softmax as the target class. We then supervise the model with cross-entropy. The Targeted Class Convergence (TCC) loss is defined as:
(TCC(0) B.
[
log
Σ. exp(fof Xx [x)) )}
exp(Jo(xx)L))
(16)
X.EB,
where ÿ = arg max = k/x;) ) is the target class. Eq. 16 defines our second objective.
4.3 Overall Objective for Test-Time Adaptation
The total loss for novel, entropy- and energy-based TTA approach, ReTTA, defined the combination of ESS (0), (T))(0), and the EM loss (f) as follows:
ERTTA (0)) E (0) + (a (f) d2 (rcc(()),
(17)
where (0) is defined in Eq. for the mini-batch B,, and A and A are the respective coefficients. In practice, (SSM (#) varies across different domains (due to various types of covariate shifts [14]) and mini-batches. Thus, balancing it with Las (0) is crucial for each adaptation. However, as the domain and data unpredictable at test time, propose self-adjusting balancing method.

---

-10g(2
Gauss
Shot
Impul
Defoc
Class
Motioo
Zoom
Snow
Prost
Contr
Blattic
Piwel
JPEG
Figure 2: Breakdown of the self-adjusting coefficient A1 during total TTA iterations on ImageNet-C (severity 5), based Table I The negative-log scale has corresponding to a 1, and higher values indicate near-zero X The four colors represent Noise, Blur, Weather, and Digital groups.
Self-adjusting coefficient. To achieve the seemingly challenging goal of self self-adjusting the bal- ance between (gs and (0), we leverage the concept of multi-objective optimization [29]. Given two objectives, the optimization problem is formulated min [o.1] []] ||a Volear( +
a)) Volssmr()]1B. a quadratic function a. The analytical solution is then given by:
(VelssM (0) V V E>)))(
( volssM(0)
(18)
of (0) VolssM(O)11
To ensure that the effect of (#) is preserved while still utilizing (f) relatively, clip
using the function (ax) = max (min(( - aa/aa, (x.11 1), 0), keeping then X remains within a practical range. This adjustment maintains stability in balancing across diverse covariate shifts. In Section we evaluate the effectiveness and versatility of ReTTA on various covariate shifts.
Experiment
We conduct experiments to validate the following: (1) the performance of ReTTA compared to existing entropy- and energy-based TTA methods under various distribution shifts, including challenging scenarios such as online label shifts; (2) the self-adjusting impact of Y₁ within the newly introduced loss &SSM (0), its projection distributions, and replacing alternative losses with SSM; and (3) the contribution of (f) to performance, its role in reducing entropy, and the sensitivity A2-
Dataset and baseline methods. We evaluate ReTTA on ImageNet-C [4], a widely-used benchmark for assessing model generalization under diverse distribution shifts. The dataset consists of 15 corruption types, divided into four main categories (Noise, Blur, Weather, and Digital), each with five severity levels, for total of 1K classes. We compare ReTTA with state-of-the-art methods
including entropy-based approaches MEMO 40 Tent 34). EATA 26 SAR 27
De 20 and
energy-based methods TEA (39 and AEA
DNN models and experimental settings. We perform experiments using two model architectures ResNet-50 (with BN/GN) and VitBase (with LN) - from torchvisi and timm, respectively. Following SAR (27). we use SGD with momentum 0.9, batch size of 64, and learning rates of 0.00025 (ResNet) and 0.001 (Vit). Unless otherwise stated, we also apply the data sampling and loss-reweighting scheme from DeYO 20 For TTA, we update only the affine parameters Oatine of the normalization bayers-batch/groutt norm in ResNet-50 and layer norm
wetBaac-followii Tent (34]. Unless otherwise stated, we fix the TCC loss coefficient at X2 1. All experiments use one-shot TTA: each test sample is observed and updated once. Further hyperparameters and implementation details are provided in Appendix B.
5.1 Robustness to Corruption in Test Data
Comparison mild scenario. Table I compares the performance of ReTTA with state-of-the-an entropy-based methods (MEMO, Tent, EATA, SAR, DeYO) and energy- based methods (TEA, AEA) ImageNet-C under mild corruption conditions (severity level 5). ReTTA achieves the highest accuracy across nearly all corruption categories, outperforming the state-of-the-an method, DeYO, in challenging noise corruptions: Gaussian (+1. +.77%) Shot (+1.8%), and Impulse (+1.8%). ReTTA also sets new accuracy benchmarks in the Blur. Weather, and Digital categories, significantly improving complex corruption such as Contrast (+1.5% and Motion Blur (+0.8%). Overall, ReTTA achieves an average accuracy of 49.2%, surpassing all compared methods by at least 0.6%, demonstrating robust and broad applicability under mild distribution shifts.

---

Mild
Clnss.
Nuise Shot
Impul.
Defoc.
Glass
Blur
Motisn
Zeam
Snow
Prost
Weather
Fog
Brtt.
Coar::
Elasse
Dipial
Pixel
1PE3
Avg.
(BN)
2.2
16.9
24.4
18.0
MEMO
7.5
27-7
25.3
28.7
32.2
61.0
37.6
23.9
Tenk
29.2
31.2
30.1
28.
27.7
41.4
494
47.2
41.5
57.7
67.4
29.2
54.8
58.5
52.4
43.1
EATA
34.9
37.1
35.8
11.4
33.0
47.1
52.7
51.6
45.7
60.0
68.1
44.4
57.9
60.6
55.1
47.8
SAR
30.5
30.6
31.3
28.5
28.5
41.9
49.4
47.1
42.2
$7.5
67.3
37.8
54.6
58.4
52.1
43.9
DeYO
35.6
37.9
37.1
33.8
34.1
48.5
52.8
52.7
46.4
60.6
68.0
46.1
58.4
61.5
55.7
48.6
TEAY
16.8
17.5
17.5
158
16.0
17.3
39.9
35.3
33.9
19.0
65.7
17.9
45.1
50.2
41.3
32.6
39.5
ReTTA (ours)
57.9 10.7 38.9 2 m0 M.1.on 49.3 53. 52.7.p:
16.1 60.7-6.1 682 47.6
58. 61.5
16.0 49.2
Table 1: Comparisons with baseline TTA methods on ImageNet-C at severity level 5 under mild
scenario in terms of accuracy (%). * 'EAA was not publicly reported and was tested directly.
Label Shifts
Guans.
Nouse Shut
Impul.
Delox
Class
Blnr
Motion
Zoom
Snow
Froot
Weather
Fog
Brit
Contr:
Elastic
Digital
Pisel
JPEG
Avg.
17.9
19.9
17.9
24.9
40.4
47.4
18.7
78.4
52.2
30.6
20.6
31.3
3.6
4.2
16.5
26.9
28.4
17.9
26.2
24.7
FATA
25.7
28.6
24.8
18.5
19.6
24.1
28.4
35.3
33.0
41.2
65.2
33.3
28.0
42.4
32.7
SAR
33.7
36.9
35.3
19.3
20.3
33.8
29.8
21.9
48.7
34,9
71.9
46.7
6.6
52.3
56.2
36.3
DEYO
42.5
44.9
43.8
22.2
15.3
41.0
13.2
52.2
$1.5
39.7
73.4
$2.6
46.9
59.3
50.3
43.9
TEA
0.4
0.4
0.4
0.2
0.1
0.4
1.2
1.1
1.3
0.44
13.5
0.5
0.3
0.3
5.0
1.7
Re TTA (ours)
42.7
45.1
44.2
29.4.no
22.9
41.1..
34.4
52.88
$8.5.1.
73.5...
(9.88.3
48.4...
59.8...3
59.3..
47.5
Villaas (LN)
9.4
6.7
3.3
2901
23.4
34.0
27.1
153
26.4
47.4
$4.7
44.0
30.5
44.3
67.6
29.9
MEMO
21.5
17.4
20.6
37.1
2906
all.5
34.4
25.0
34.8
55.2
08.10
54,9
37.4
55.5
57.3
39.1
Tost
33.9
1.8
27.2
$4.8
$2.9
58.6
54.3
12.4
11:7
69.7
76.3
66.3
59.6
60.7
66.6
47.7
EATA
36.2
34.7
35.5
43.4
44.3
(9.3
88.5
53.2
53.5
62.3
72.7
18.8
58.0
61.7
62.8
49.2
SAR
42.5
34,9
44.1
50LD
50.5
55.6
53.1
59.7
47.2
66.2
75.2
50.5
640.1
67.3
65.0
54.8
D=YO
53.5
36.0
54.6
57.6
58.7
63.7
462
67.6
66.0
73.2
77.9
66.7
69.0
73.5
70.3
62.3
6.9
25.7
54.0
50 64.7
.0.1 58.5
69.0
.0 71.2 77.9
67.6
20.4
g01 71.6.u3 -3
Table 2: Comparisons with bascline TTA methods on ImageNet-C (severity 5) under online label
shifts (imbalance ratio=m)) in accuracy (%).. * 'TAA was not publicly reported and was tested directly.
In this evaluation, Figure illustrates the impact of the self- adjusting coefficient A₁ on corruption-seccffic performance when applying SSM. The influence of SSM is most pro- nounced in the Noise category, where it is fully utilized and also shows to be essential in challenging corruptions like Frost and JPEG. Interestingly, in a more difficult corruption case, such as the Contrast, we observe that SSM is used con- servatively, allowing the TCC to adjust the impact of EM principally. This self-adjustment signifies the importance of the adaptive balance between entropy and energy.
4.0
EM
Entropy (H(p))
SSM
3.5
EM SSM + TCC
3.0
2.5
2.0
0
150
350
550
750
TTA iterations
Figure 3: Effects of TCC (Defocus)
Comparison online label shifts. We evaluate ReTTA
under severe online label shifts, following the setting of an infinite imbalance ratio (pmaa(y)) ())) (0) in SAR. Table presents the results for this challenging scenario, highlighting ReTTA's robustness on both ResNer-50 (GN) and VitBase (LN). On ResNer-50 (GN), ReTTA significantly outperforms the state-of-the-art method, DeYO, across difficult corruptions, notably Defocus (+7.2%). Zoom (+21.2%) and Fog (+18.8%). achieving an overall improvement of 3.6% in average accuracy. Similarly, for VitBase (LN), ReTTA surpasses Yo on almost all corruption categories, with notable improvements on Impulse Noise (+0.69 (4066)) Zoom Blur (+ 12.3%)) and Pixel (+0.6%), resulting in an average accuracy gain of 2.5%. These results demonstrate ReTTA's outstanding robustness to label distribution shifts.
5.2 Ablation Study
Effects of the balancing parameter Xg. Figure 4(a) shows how varying X2 impacts ReTTA's accuracy. Performance peaks at A22 = which we adopt for all experiments; deviations in either direction degrade accuracy, showcasing the importance of a well-tuned TCC coefficient. Since TTA unsupervised, overly large X2 can be detrimental. Figure 3 further demonstrates that with A ReTTA reduces entropy while boosting accuracy over EM 20 on the Defocus corruption (Table D
Effects of alternatives for SSM. Figure 4(b) shows the impact of SSM alternatives, including Score Matching (SM) and Sliced Score Matching with Variance Reduction (SSM-VR), which applies when p( v) follows a Normal distribution [33]. While SSM, our chosen loss, outperforms SM (which
shows marginal gains), it performs slightly better than SSM-VR, with an average difference of approximately 0.1% 9. This demonstrates SSM's effectiveness within RTTAA
8

---

Accuracy (%)
49.2
49.20
Accuracy (%
49.2
40.10
Accuracy (%)
49.2
40335
40.12
49.00
49.0
49.0
49.0
48.8
48.00
48.8
48.8
0.25
SSM-VR
Normal Rademacher Sphere
A2
Alternative loss
p(v)
(a) Effects of varying Ae
(b) Effects of alternatives for SSM
(c) Effects of varying p(v)
Figure 4: Effects of varying components in ReTTA: Xe for balancing TCC, alternative losses for
SSM, and projection vector distributions within SSM. Experimental settings match those Table
Label Shifts
Canss.
Nouse
Shot
Insuu.
Defoc.
Glass
Blur
Motion
Zoom
Snow
Frost
Weather
Fog
Bri..
Cost::
Blasti
Digizz
Pisel
JPEG
Ave.
ResNes50 (CN) FM
X0
42.5
44.9
43.5
22.2
16.3
41.0
13.
52.2
51.5
39.
73.4
52.6
46.9
50.3
50.1
43.9
+SSM
42.2
44.5
45.8
26.7
22.3
40.3
13.4
51.6
50.8
58.4
73.1
49.8
45.3
59.5
59.1
TTC
59.5
59.5
LICETIA
22.9
49.8
59.8
Vilaac (LN) FM
51.5
36.0
54.6
57.6
56.7
63.7
46.2
67.6
66.1
732
773
66.7
6930
733
20.3
62.3
+SMM
$4.1
55.0
55.4
51.8
58.4
64.7
59.2
69.0
67.0
71.4
71.0
67.4
69.4
74.1
70.8
64.7
FTCC
53.9
54.8
55.0
57.8
58.1
64.4
41.8
68.1
66.7
71.1
77.9
67.1
0910
73.9
09.3
03.3
&RETTA (Bg
34.0
55.0
55.2
57.8
58.7
64.7
58.5
68.0
67.1
71.2
77.9
67.6
69.8
74.1
71.6
64.8
Table 3: Effect of components in ReTTA. Each denotes accuracy (%) on ImageNet-C (severity 5)
under online label shifts (imbalance ratio oc), with Ye as the baseline EM method.
Naise
Blur
Weather
Dightd
Gouss.
Impui.
Defoc.
Glass
Mation
Zoom
Snow
Foz
Brit.
Cocar:
Elastic
Pixel
JPDG
ResNet-50 (BN)
EM
34.9
37.
35.8
33.0
47.1
31.6
50.0
68.
44.4
60.6
R.RTTAA
35.1
$1.7
44.7
60.7
Reshet (0
EM
27
30.5
30.5
31.3
28.5
28.5
41.9
49.4
47.1
42.2
57.5
67.3
37.8
54.6
53.4
52.1
43.9
LReTTA (Es,
31.8
34.1
32.7
27.8
27.9
44.1
58.7
48.3
42.4
53.5
67.7
40.5
55.4
59.2
53.2
44.9
ResNes-SUl
BM
200
35.5
37.9
37.1
35.8
34.1
45.5
52.8
52.7
46.4
50.6
68.0
46.1
55.4
61.5
53.7
43.6
L.ReTTA (Fo
37.3
39.7
38.9
34,5
34.1
49.3
53.1
52.7
06.1
68.7
68.2
47.6
58.6
61.5
56.0
49.2
Table 4: Adaptivity of components ReTTA applied to state-of-the-art EM methods (EATA, SAR,
and YO).). Each denotes accuracy (%) ImageNet-C (severity 5) under mild scenarios.
Effects of projection distributions. Figure(4() shows the effect of different projection distributions, Rademacher {=1) and the uniform over the hypersphere (Sphere). Normal is our chosen distribution. The minimal performance variation suggests that the choice of projection distribution has little impact on accuracy, demonstrating ReTTA's insensitivity different projections.
Effects of components in ReTTA Table 3 shows that integrating SSM improves performance,
especially for ResNet-50 (GN), with additional gains from incorporating TCC. The combination of SSM and TCC outperforms the baseline EM method, demonstrating that ReTTA's integration of entropy and basegy-based optimization provides robust, general- purpose solution. This approach excels across various distribution shifts, particularly under challenging online label shifts, as also reflected in improvements with VitBase (LN).
Effects of SSM and TCC in state-of-the-arr EM methods. Table shows that combining SSM and TCC into EATA, SAR, and DeYO yields accuracy gains across most corruption types. De sees its largest gain in the Noise category, while SAR outperforms its baseline on all but Defocus and Glass corruptions. EATA shows marginal dains-speculativee because built-in forgetting mitigation dampens the impact of deviated updates from the original parameters. While gains for Defocus and Glass are more modest overall, integrating energy-based and class-targeted components in ReTTA effectively strengthens performance to diverse distribution shifts.
5.3 Case Study
Potential in test-time domain adaptation. We further evaluate ReTTA under mild adaptation conditions on three additional ImageNNet-calle cuttof-distribuiin benchmarks. ImageNet-R contains rendered versions of ImageNet objects, introducing large domain shift. In contrast, ImageNet is a closely related re-sampling of the original ImageNet distribution, while ImageNet-S consists of single-channel sketch drawings. As shown in Table 5 ReTTA improves accuracy to 47.4% (ResNet- 50) and 61.7% (VitBase) on ImageNet-R, comparable its gains on ImageNet-C. On ImageNetV2, ReTTA mitigates the performance degradation often observed with EM-based methods, which tend

---

ImageNet-R
ResNet-50 (GN)
VitBase (LN)
ImageNetV2
ImageNet-S
No adapt.
40.8
50.9
ResNet-SO
63.20
24,09
Tent
42.8
55.3
Tent
63.07
30.50
TEA
7.0
22.9
TEA
57.28
8.87
EAIA
41.9
51.1
EAIA
63.14
35.24
RcTTA (Algorithm 1)
42.4
52.1
LReTTA (Algorithm 1)
63,20
35.24
SAR
41.9
$1.8
SAR
63.06
31.74
L.RTTA (Algorithm 2)
42.3
52.5
LRcTTA (Algorithm 2)
63.14
32.16
DeYO
47.0
61.3
DeYO
62.89
35.83
LReTTA (Algorithm 3)
47.4
61.7
L.ReTTA (Algorithm 3)
63.06
35.86
Table Comparison of ReTTA with baseline TTA methods defined by Algorithms 1-3 in Appendix B.1 on three ImageNet-scale nitoof-distribution benchmarks under mild adaptation settings. Each result reports accuracy (%) on ImageNet-R (rendered objects), ImageNetV2 (re-sampled validation distribution), and ImageNet-S (sketch-style grayscale images) using ResNet-50 and VitBase.
Weather
Digial
Impul.
Fog
Aug.
EATA 26 (Sourve)
73.8
73.
13.6
73.7
73.5
13.8
74.1
74.1
74.6
74.9
74.1
74.2
74.3
74.0
14,0
LReTTA (Socace))
73.4
73.5
73.3
73.6
73.4
74.0
74.1
74.1
74.0
74.4
74.7
74.0
74.1
74.4
73.9
73.9
EATA
26 (Lifeloogg)
28.3
38.7
28.0
34.1
34.0
42.1
32.9
50.9
45.6
5938
679
44.1
57.4
60.1
54.9
48.3
LReTTA
(1 (ifdvegg
36.0
388
38.2
342
338
47.4
53.1
514
45.
59.9
68.
44.6
57.5
60.7
54.9
48.3
Table 6: Adaptivity of ReTTA when integrated with EATA (Algorithm 1) under the lifelong adaptation setting, following the continual adaptation strategy described [26]. Each result represents accuracy (%) on ImageNet-C (severity 5) using ResNet-50 (BN) with a source accuracy of 76.13% "Source" denotes the validation performance on ImageNet- k measured during the lifelong adaptation process.
to incur uncertainty- -indindy- penalties in similar or overlapping domains. On ImageNet-S, ReTTA maintains the top rank among baselines, although the limited grayscale information and reduced modes of distribution likely constrain the influence of SSM.
Potential in lifelong adaptation. Building on Lemma 3 we speculate that SSM can also mitigate forgetting. During each adaptation step, the "negative phase" slightly raises energy for samples from the source distribution, while the "positive phase" lowers energy for newly corrupted samples. Rather than offsetting each other within single SGLD update, these two forces may reach a balanced state that broadens the source density to cover new data without overwriting its original support. Table/[]confimm that integrating ReTTA with EATA preserves similar gains in lifelong protocols as in standard TTA (+0.2%). In particular, ReTTA+EATA achieves an average improvement of +0.3% in the lifelong setting, yielding a final source accuracy of 73.9%. These results demonstrate that ReTTA complements EM not only by enhancing adaptation under distributional shifts but also by maintaining source performance during extended lifelong adaptation.
Conclusion
This paper questions the sufficiency of minimizing entropy alone for effective TTA. We identify two key obstacles entropy minimization under distribution shifts: the inability to estimate the test data distribution and the failure enhance the model's 's discriminability further. This study shows that simultancous entropy-cnerge minimization is one goal-dra goal-driven approach to overcoming these problems. We propose ReTTA, which combines SSM and TCC for energy reduction and discriminability. By adaptively balancing both objectives, ReTTA offers a scalable solution to distribution shifts. Extensive experiments demonstrate that ReTTA outperforms existing entropy- energy- -aaed TTA methods.
Acknowledgments and Disclosure of Funding
This research was partly supported by the IITP(Institte of Information & Communications Technol- ogy Planning & Evaluation) - (Infoommattioonnn Technology Research Center) grant funded by the 50%), the National Research Foundation of Korea(NRF) grant funded by the Korea goV))) (RS-2025-00562437, 40%), and Insti- tute of Information & communications Technology Planning & Evaluation (ITPP) grant funded by the Korea government(MSSr) 2000S-2022-00155591 Artificial Intelligence Convergence Innovation
Korea (SVern (ITT 2-----S-2002-0002
Human Resources Development (Kyung Hee University), 10%).
10

---

References
[1] Yeong Hak Bang. Yoon Ho Choi, Mincheol Park, Soo-Yong Shin, and Scok Jin Kim. Clinical relevance of deep learning models predicting the onset timing of cancer pain exacerbation. Scientific Reports, 2023.
[2] Malik Boudiaf, Romain Mueller, Ismail Ben Ayed, and Luca Bertinetto. Parameter-free online test-time adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
[3] Wonjeong Choi, Yeon Kim, Jungwuk Park, Jungmoon Lec, Younghyun Park, Dong- Jun Han, and Jackyun Moon. Adaptive energy alignment for accelerating test-time adaptation. In International Confer- ence on Learning Representations 2025.
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models. Advances in Neural Information Processing Systems, 2020.
[5] Yilun Du, Shuang Li, Joshua Tenenhaum, and Igor Mordatch. Improved contrastive divergence training of energy-based models. In International Conference on Machine Learning, 2021.
[6] Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl- Sohl-Diccstten, Arnad Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. In International Conference on Machine Learning, 2023.
Zhekai Du, Jingjing Li, Lin Zuo, Lei Zhu, and Ke Lu. Energy-based domain generalization for face anti-spoofing. In Proceedings of the 30th ACM International Conference on Multimedia, 2022.
[8] Taesik Gong, Jongheon Jeong. Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. NOTE: Robust continual test-time adaptation against temporal correlation. Advances in Neural Information Processing Systems, 2022.
[9] Tacsik Gong, Yewon Kim, Tacckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. Sotta: Robust test-time adaptation on noisy data streams. Advances Neural Information Processing Systems, 2023.
[10] Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and Zico Kolter. Test time adaptation via conjugate pseudo-labels. Advances Neural Information Processing Systems, 2022.
[11] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems, 2004.
[12] Will Grathwohl, Kuan-Chitn Chinhhhihh Wang, Kevin Swersky. Your classifier is secretly energy based model and you should treat like one. In International Conference on Learning Representations, 2020.
Jacobsen, David Duvenaud, Mohammad Norouzi, and
[13] Yue He, Xinwei Shen, Renzhe Xu, Tong Zhang, Yong Jiang, Wenchao Zou, and Peng Cui. Covariate- shift generalization via random sample weighting. In Proceedings of the AAAI Conference on Artificial Intelligence. 2023.
[14] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[15] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems, 2021.
[16] Kexin Jin, Chenguang Liu, and Jonas Latz. Subsampling error in stochastic gradient langevin diffusions. In International Conference Artificial Intelligence and Statistics, 2024.
[17] Lingdong Kong, Xiang Xu, Jiawei Ren, Wenwei Zhang, Liang Pan, Kai Chen, Wei Tsang Ooi, and Ziwei Liu. Multi-modal data-efficient 3d scene understanding for autonomous driving. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[18] Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, Fujic Huang, et al. A tutorial on energy- based learning. Predicting Structured Data, 2006.
[19] Dacun Lee, Jachong Yoon, and Sung Ju Hwang. BECOTTA: Input-dependent online blending of experts for continual test-time adaptation. In International Conference on Machine Learning, 2024.
[20] Jonghyun Lee, Dahuin Jung, Sachyung Lee, Junsung Park, Juhyeon Shin, Uiwon Hwang, and Sungroh Yoon. Entropy is not enough for test-time adaptation: From the perspective of disentangled factors. In International Conference Learning Representations, 2024.
11

---

[21] Buyu Li. Wanli Ouyang, Lu Sheng, Xingyu Zeng, and Xiaogang Wang. Gs3d: An efficient 3d object detection framework for autonomous driving. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2019.
[22] Chen Li and Yoshihiro Yamanishi. Gxvaes:
Two joint vaes generate hit molecules from gene expression profiles. In Proceedings of the AAAI Conference on Artificial Intelligence. 2024.
[23] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, 2020.
[24] Fu Lin, Rohit Mittapalli, Prithvijit Chattopadhyay, Daniel Bolya, and Judy Hoffman. Likelihood landscapes:
A unifying principle behind many adversarial defenses. In Computer Vision-ECCV 2020 Workshops: Glasgow, UK. August 23-28, 2020, Proceedings. Part 16. 2020.
[25] James Martens, Ilya Sutskever. and Kevin Swersky. Estimating the hessian by back-propagating curvature. In International Conference on Machine Learning, 2012.
[26] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang. Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Conference on Machine Learning. 2022.
[27] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations, 2023.
[28] Steffen Schneider. Evgenia Rusak, Luisa Eck, Oliver Bringmann. Wieland Brendel, and Matthias Bethge. Improving robustuess against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems. 2020.
[29] Ozan Sener and Vladlen Koltun. Multi-task learning multi objective optimization. Advances in Neural Information Processing Systems, 2018.
[30] Junha Song. Jungsoo Lee, In So Kweon, and Sungha Choi. Ecotta: Memory-efiie efficient continual test-time adaptation via self distilled regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, 2023.
[31] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 2019.
[32] Yang Song and Diederik P Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.0320 2021.
[33] Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, 2020.
[34] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
[35] Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[36] Zehao Xiao, Xiantong Zhen, Shengcai Liao, and Cees G. M. Smoek. Energy- test sample adaptation for domain generalization. In International Conference on Learning Representations, 2023.
[37] Binhui Xie, Longhui Yuan, Shuang Li, Chi Harold Liu, Xinjing Cheng, and Guoren Wang. Active learning for domain adaptation: An energy- -base approach. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
[38] Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 15922- 15932, 2023.
[39] Yige Yuan, Bingbing Xu, Liang Hou, Fei Sun, Huawei Shen, and Xueqi Cheng. Tea: Test-time energy adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.
[40] Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmentation. Advances in Neural Information Processing Systems, 2022.
[41] Han Zou, Jianfei Yang, and Xiaojian Wu. Unsupervised energy-based adversarial domain adaptation cross-domain text classification. In Findings of the Association for Computational Linguistics; ACL- IJCNLP 2021, 2021.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately reflect our theoretical empirical contribu- tions, including the proposed solutions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be pereeived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation long it is clear that these goals attained by the paper.
2. Limitations
Question: Does paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Appendix.
Guidelines:
The answer NA means that paper has no limitation while answer means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust results are violations these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
grounds for rejection,
Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The paper elearly states all theoretical assumptions and provides complete proofs.
Guidelincs:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
13

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
Formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Results and experimental settings are clearly documented in the paper and Appendix. Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For novel architecture, describing the architecture fully might suffice,
example, if the contribution the contribution is specific model empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the of large language model), releasing of model checkpoint, or other means that are appropriate to research performed.
While NeurIPS does not require relcasing code, the conference does require submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe architecture clearly and fully.
(c) If the contribution a way to access this model for reproducing the results a way to reproduce model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
new model (e.g.. large language model), then there should either be
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome to describe the particular way they provide closcd-sourc source models, it may be that access to the model is limited some way (c.g., to registered users), but it should be possible for other rescarchers to have some path to reproducing or verifying the results.
reprodueibility. In the case of
Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to
faithfully reproduce the main experimental results, described supplemental material?
Answer: [No]
Justification: Code will be released after the review process avoid potential violations of the double-blind policy.
Guidelines:
The answer means that paper does not include experiments requiring code.
Please see the NeurTPS code and data submission guidelines guides/Codenn SwwodeSuaiseeionoo..
http://nippscccppulp/pp
for more details.
While we encourage release of code and data, we understand that might not possible,
so "Na" an acceptable answer. Papers cannot rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source henchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: : :/nnis///////////////::::eee cc/public/ muiidess/CooeeSttmleesssoooooo.. for more details.
14

---

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reprodueible, they should state which ones are omitted from script and why.
At submission time, to preserve anonymity, authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
Details of the experimental settings are included the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report error bars (standard deviations) with three random trials.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors variability that the error bars are capturing should be elearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap. etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 2-sigma enror bar than state that they have 96% CI. if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should he careful symmetric error hars would yield results that are out of range (c.g. negative error rates).
show in tables figures
error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [No]
Justification: do not report specific compute details, such as workers, memory, and time, as resource allocation varies depending on data center schedules and shared usage.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
15

---

The paper should provide the amount of compute required for each of the individual experimental well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn 't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform. every respect, with NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
No harmful content or ethical risks are involved.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain special circumstances that require deviation from the Code of Ethics.
authors should make sure to preserve anonymity (e-g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Societal impacts are discussed in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will he foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example. it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms could arise when technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from feedback time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place responsible release data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification:
No use of the models and data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example hy requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
16

---

We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: datasets and models are used compliance with their licenses.
Guidelines:
The answer means that the paper does not use existing assets.
authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset is used and, possible, include URL.
The name of the license (e.g.. g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g.. website), the copyright and terms of service of that source should he provided.
If assets released, the license, copyright information, and terms of the package should be provided. For popular datasets, paaperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification:
No assets introduced in this work.
Guidelines:
The NA means that the paper does release
Researchers should communicate the details of the dataset/code/model part of their sub- missions structured templates. This includes details about training. license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Not applicable human subjects crowdsourcing involved.
Guidelines:
answer NA means that paper does not involve crowdsoureing nor research with human subjects.
Including this information in supplemental material is fine, but main contribution of paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants. whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
17

---

Justification:
Not applicable - human subjects crowdsoureing involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review,
Declaration LLM usage
Does the paper describe the usage of LLMs if it is important. original. or non non-standard component of the core methods in this research? Note that if the L.M is used only for writing, editing, formatting purposes and does impact the core methodology, scientific rigorousness, originality of the rescarch, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not used the core methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM should or should not be described.
(https: http:///aaarpp.ce//oooo////er/// for what
18


Reviews:
[{"review_id": "bvLG4c6deD", "reviewer": "Reviewer_GM3k", "summary": "The paper revisits test-time adaptation (TTA) through the lens of energy–entropy duality, showing that classic entropy minimization can stall in high-energy regions and thus fails to maximise test-sample likelihood. To overcome this gap, the authors introduce ReTTA, which couples entropy minimisation with (i) a sampling-free Sliced Score Matching energy loss and (ii) a Targeted Class Convergence cross-entropy loss, balanced by a self-adjusting coefficient. Extensive experiments on ImageNet-C with both ResNet-50 and ViT-Base demonstrate the effectiveness of the proposed method.", "questions": "Please refer to the Weaknesses.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Positive Points:\n1. The energy–entropy duality analysis pinpoints why entropy minimisation alone is insufficient and directly motivates the SSM and TCC objectives.\n2. ReTTA is tested across two architectures, two normalisation schemes and multiple corruption and label-shift scenarios, consistently surpassing state-of-the-art baselines.\n\nNegative Points:\n1. The Sliced Score Matching component introduces second-order Jacobian terms, yet the paper provides no comparison of runtime or memory consumption against standard entropy minimisation (EM) or energy-based TTA baselines such as TEA.\n2. Entropy minimization may encounter the overconfindence issues[1]. The proposed Targeted Class Convergence loss adopts the current top-1 output as a pseudo-label, which could amplify this mis-calibration and accumulate errors under heavy domain shift; the manuscript neither analyses the phenomenon nor proposes safeguards.\n3. The influence of the λ₁ weight on SSM, as well as the dynamics of the self-adjusting coefficient, are not ablated, leaving their robustness and default settings unclear.\n4. While the method is evaluated on ViT-Base under severe ImageNet-C corruption, results for the same architecture in the mild setting are absent, limiting evidence of effectiveness across shift intensities.\n\n[1] come: test-time adaption by conservatively minimizing entropy. ICLR 2025", "was_revised": true, "final_justification": "I appreciate the authors’ detailed responses. The authors have addressed most of concerns, such as more experiments, more clarification about the overconfidence. However, the memory overhead and inference latency remain relatively high.\n\nThat said, considering the overall contribution and the improvements made, I will increase my score.", "rebuttal": "We appreciate your valuable time and effort in providing constructive feedback. We believe that clarifications in response to the concern raised enhance the quality of this manuscript and hope for more favorable consideration of our research.\n\n&nbsp;\n\n>W1. The Sliced Score Matching component introduces second-order Jacobian terms, yet the paper provides no comparison of runtime or memory consumption against standard entropy minimisation (EM) or energy-based TTA baselines such as TEA\n\nWe appreciate the reviewer’s careful feedback. While Tent (EM-only) incurs minimal overhead, energy-based approaches generally involve higher memory and runtime demands. For example, TEA, which applied only an energy‑based contrastive divergence (CD) loss, requires approximately 8 GB of memory and exhibits significant runtime due to its generative procedures. In comparison, our combined EM + SSM implementation uses around 10 GB, showing that while the energy‑based method is inherently more costly than EM, our SSM adds only a modest overhead. Notably, our method runs about 30% faster than TEA, thanks to optimized Jacobian/Hessian computations that outperform TEA’s generative CD updates on batch data.\n\n| Method            | Peak GPU Memory (MB) | Runtime (s) |\n|-------------------|-----------------:|-----------------:|\n| **Tent (EM)**     |              2863 |            173 |\n| **TEA (Energy‑only CD loss)** |    8136  |            **3553** |\n| **Ours (EM + SSM)** |          **10439**  |            2480 |\n\n_Tested on: Intel(R) Xeon(R) Silver 4214 @ 2.20GHz CPU, a single NVIDIA TITAN RTX 24GB GPU_\n\nWe emphasize that this work prioritizes algorithmic contributions as a foundation for future optimizations. As energy-based TTA continues to mature, we expect engineering advances—such as memory-efficient formulations, faster approximations, and improved system-level support—to follow naturally.\n\n\n&nbsp;\n\n>W2. The proposed Targeted Class Convergence loss adopts the current top-1 output as a pseudo-label, which could amplify this mis-calibration and accumulate errors under heavy domain shift; the manuscript neither analyses the phenomenon nor proposes safeguards\n\nWe did not intend TCC to serve as a standalone approach. Rather, we were motivated that EM alone could not guess a test sample’s neighborhood or accurately estimate its likelihood, so we first applied SSM to pull samples (and their neighbors) into high‑likelihood regions. We could then use TCC to sharpen class probabilities on these refined representations. By integrating our losses into state-of-the-art EM workflows, we also inherited an entropy-based data filtering step, adapting only to low-entropy (and thus highly reliable) samples, to naturally guard against error amplification. In this design, SSM provided the core likelihood safeguard, data filtering prevented error accumulation, and TCC delivered the final discriminative push. As Table 3 shows, coupling EM with entropy‑based data filtering alone yielded dramatic improvements of 7% or 20% on corruptions like Glass or Frost, and on average delivered over a 1% gain versus the EM method. Also, the full ReTTA with SSM produced an average increase of over 2.5% relative to the EM method. It confirms that these components work in concert for robust, strong adaptation performance.\n\n&nbsp;\n\n>W3. The influence of the $\\lambda_1$ weight on SSM, as well as the dynamics of the self-adjusting coefficient, are not ablated, leaving their robustness and default settings unclear\n\nWe note that our self‑adjusting coefficient is exactly $\\lambda_1$, which by design adapted automatically to any distribution shift—so a fixed‑value ablation was unnecessary. As shown in Figure 1, the bar plot tracked $\\lambda_1$ across different corruption types and batch iterations, demonstrating how it dynamically scaled the SSM gradient without a manual option.\n\n&nbsp;\n\n>W4. While the method is evaluated on ViT-Base under severe ImageNet-C corruption, results for the same architecture in the mild setting are absent, limiting evidence of effectiveness across shift intensities.\n\nTo ensure fairness, we used the DeYO source code repository to run the full suite of 15 severity‑5 corruption evaluations on ViT‑Base (LayerNorm) ourselves. The complete results appear below. Even under this mild setting, ReTTA led on average, achieving the highest accuracy on 10 of the 15 corruptions and second‑best on the remaining five. This confirmed that, across different model architectures, under severe distribution shifts, the simultaneous contributions of energy minimization and discriminability were crucial. Due to time constraints, we could not extend this evaluation to all five severity levels; instead, we performed additional tests on ResNet‑50‑BN at severities 1 and 3 (for Reviewer 34Nx), which mirrored these performance trends and further validated ReTTA’s robustness across varying shift intensities. We observed that at lower severities, all TTA methods delivered similar average performance (with ReTTA only marginally better), whereas as severity increased, jointly minimizing energy became essential to sustain high accuracy.\n\n| Method          | Gauss. | Shot  | Impul. | Defoc. | Glass | Motion | Zoom  | Snow  | Frost | Fog   | Brit. | Contr. | Elastic | Pixel | JPEG  | Avg  |\n|-----------------|-------:|------:|-------:|-------:|------:|-------:|------:|------:|------:|------:|------:|-------:|--------:|------:|------:|-----:|\n| No adapt        |   35.1 |  32.2 |   35.9 |   31.4 |  25.3 |   39.4 |  31.6 |  24.5 |  30.1 |  54.7 |  64.5 |   49.0 |    34.2 |  53.2 |  56.5 | 39.8 |\n| Tent            |   51.9 |  51.5 |   53.1 |   51.8 |  47.8 |   56.3 |  49.4 |  10.6 |  18.3 |  67.1 |  73.3 |   66.7 |    51.5 |  65.2 |  64.5 | 51.9 |\n| EATA            |   **55.9** |  **56.4** |   **57.1** |   53.3 |  53.5 |   58.2 |  **58.5** |  61.9 |  60.1 |  71.4 |  75.3 |   **68.5** |    62.4 |  69.5 |  66.6 | 61.9 |\n| SAR             |   51.8 |  51.7 |   52.9 |   50.8 |  48.9 |   55.6 |  49.1 |  12.8 |  51.0 |  65.8 |  73.1 |   66.1 |    51.9 |  64.0 |  63.3 | 53.9 |\n| DeYO            |   54.5 |  55.1 |   55.8 |   53.8 |  54.6 |   62.4 |  58.0 |  64.0 |  63.3 |  71.6 |  77.3 |   67.3 |    65.6 |  71.5 |  68.3 | 62.9 |\n| TEA             |    8.8 |  20.2 |   11.8 |    1.9 |   4.2 |   14.7 |   4.4 |   1.2 |   3.6 |   8.9 |  73.4 |   62.8 |     3.0 |  67.3 |  64.2 | 23.3 |\n| **ReTTA**       |   55.1 |  55.9 |   56.5 |   **56.4** |  **55.9** |   **62.5** |  57.9 |  **64.6** |  **63.5** |  **72.2** |  **77.4** |   67.4 |    **65.7** |  **71.6** |  **68.6** | **63.4** |\n\nWe hope these clarifications have effectively addressed your concerns and provided a clearer understanding of our contributions."}, {"review_id": "CPfcAbSupR", "reviewer": "Reviewer_YKNW", "summary": "The paper presents a test time adaptation framework called ReTTA that focuses on joint entropy and energy minimization. For this, it first studies the dual relationship between entropy and energy and shows that minimization of entropy alone does not ensure minimization of energy (or maximization of likelihood of the test data). Hence, both need to be explicitly minimized. Based on this, they propose ReTTA’s objective: EM (entropy minimization) + SSM (energy minimization) + TCC (for driving entropy to zero). \n\nFor evaluation, they consider ImageNet-C and two neural network architectures: ResNet-50 and ViT. The results show that their entropy+energy based approach outperforms SOTA entropy-only or energy-only based approaches.", "questions": "This paper presents results on only ImageNet-C. However, a list of datasets for TTA experimentation (covering various domains and problems) can be found at https://docs.google.com/spreadsheets/d/10tOlFDA5hLSpyv5Wv8zRcXSbUEDLfxP-YhU82AZvYJo/edit?pli=1&gid=0#gid=0\n1. How well will ReTTA generalize to other datasets? \n2. Is the method directly applicable to other domains such as sequential text or speech? \n3. If no, will it require domain or problem-specific modifications?\n\n**Minor comments:**\n1. Fig 1d is not described in the caption. \n2. In Table 1, same highest values for DeYO are not in bold. Similar observation in Table 3 and Table 4.\n3. It might be helpful for the reader if entropy minimization and energy minimization could be differentiated as EntM and EnM, respectively. I had to look up multiple times whether EM referred to entropy or energy minimization.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. Originality: the paper highlights the need to minimize both entropy and energy for TTA, providing new insights on this topic. Previous methods have either focused on entropy minimization or energy minimization but not on both together. Based on this, the paper presents a new objective function that combines entropy and energy minimization for TTA, including an auto-adjusting coefficient to balance the two objectives.\n2. Significance: the results are important for the research community because, based on the insights and results presented in this study, researchers will likely focus more on the joint minimization of entropy + energy, possibly deriving new methods for TTA.\n3. Quality: the paper is technically good, with claims and motivation for the proposed objective function well supported by theoretical (Lemma 1 and Lemma 2, Theorem 1) and experimental analysis (Fig 1d) on the relationship of entropy and energy minimization. Although shown for only one dataset, experimental results show the effectiveness of the proposed objective under various distribution shifts, including the extreme case of online label shifts. The study is comprehensive covering additional aspects such as computational overhead, impact of hyperparameters and their sensitivity. Limitation of the proposed method (for the streaming case) has also been discussed.\n4. Clarity: the paper is well written and clearly organized, with the required flow and necessary information needed for a reader’s clear understanding. Authors do a very good job of presenting the research story starting with the motivation (derived for the study of relationship between entropy and energy), followed by the proposal of different components of the objective function. \n\n**Weaknesses:**\n1. Unlike baseline methods, such as Tent (3-4 datasets), EATA (3 datasets), DeYO (5 datasets), this paper presents experiments on only ImageNet-C. There is no study or discussion on how well ReTTA will generalize to other datasets and domains (text, speech).", "was_revised": true, "final_justification": "Based on my reading and other reviews, I found the proposed method to be novel and useful for the research community. The motivation is theoretically grounded and the effectiveness of the method has been verified empirically. My main concern was related to the method's generalization to other image-based datasets, which the authors have addressed with additional experiments during the rebuttal. The paper is overall well written and clear to read as well. I therefore recommend an \"Accept\" for this paper.", "rebuttal": "We appreciate your valuable time and effort in providing constructive and positive feedback. We have clarified the remaining points on Q1-Q3 below and added responses and a preliminary case study in text to show how ReTTA can extend beyond images.\n\n&nbsp;\n\n>Q1. Generalization to other image datasets] \n\nReTTA’s three pillars, Entropy Minimization (EM), Sliced Score Matching (SSM), and Targeted Class Convergence (TCC), are fully modular and plug-and-play for image classification. In practice, once a model (e.g., CNN or ViT) is trained:\n1. Entropy: We can compute the entropy loss $-H(p)$ on images and take a gradient step in model parameters.\n2. SSM: We only need the pixel-space gradient, which any differentiable model naturally provides.\n3. TCC: We then apply a cross-entropy push toward the most probable class.\n\nTherefore, no extra parameters and no architectural tweaks are required to move from ImageNet-C to variants like ImageNetV2, ImageNet-R, ImageNet-S, or other image datasets. We covered evaluations on these ImageNet variant datasets in response to Reviewer 34Nx.\n\n&nbsp;\n\n>Q2 and Q3. Applicability to text or speech modalities\n\nWhile pixel-space gradients make SSM straightforward for images, raw text and audio waveforms lack such a grid. Directly computing score and Hessian on token IDs or raw waveforms may not capture meaningful structure. Instead, one should:\n1. Choose a representation (e.g., token embeddings, spectrogram frames).\n2. Compute then the Jacobian and Hessian trace in that representation space.\n3. Formulate SSM based on those derivatives.\n\nTo show how this works in text, we conducted an exploratory sentiment classification study on Amazon Polarity Reviews.\n\n&nbsp;\n\nA. Data, Model, and Task\n\nWe evaluated ReTTA on a cross-domain text classification scenario using the Amazon Polarity Reviews dataset. The source domain was Books, and the target domain was Electronics. Each review carried a 1–5 star rating; we binarized these into negative (ratings 1–2 $\\rightarrow$ label 0) and positive (ratings 4-5 $\\rightarrow$ label 1), discarding neutral reviews with rating 3. At test time, we deployed a Books-pretrained model to classify Electronics reviews under the TTA paradigm, using DistilBERT [1] from HuggingFace.\n\n&nbsp;\n\nB. Implementation of SSM in Embedding Space\n\nTo extend SSM to text, we treated each input as the token-embedding sequence $s\\in \\mathbb{R}^{l\\times 768}$ ($l$=sequence length) produced by DistilBERT’s encoder. For each $s$, we computed the Jacobian $\\nabla_{s}\\log p(s)$ and Hessian $\\nabla^2_{s}\\log p(s)$ via differentiation with the energy-based models with respect to $s$, i.e., $p(s):=\\exp(-E_\\theta(s))/Z(\\theta)$. We then sampled random projection vectors $v\\in\\mathbb{R}^{l\\times 768}$ and minimized the SSM loss: \n\n$\\mathbb{E}_{p(v)}\\left[ v^\\top \\nabla^2_s \\log  p(s) v + \\frac{1}{2} || v^\\top \\nabla_s \\log  p(s) ||^2 \\right]$.\n\n&nbsp;\n\nC. Experimental settings for text classification\n\nWe used the AdamW with a base learning rate scaled linearly by batch size, specifically setting $2e-5\\times(\\text{batch size} / 32)$ and a batch size of 256. The training epoch was set to 3. For Books training, we retrained all transformer layers and the classifier heads, keeping embeddings frozen. For Electronics TTA, we only updated the LayerNorm affine weights in transformer layers. Sequences were truncated/padded to 128 tokens; Books were split 90%/10% train/validation. Given that we addressed a single source to target shift, we did not employ a self‑adjusting coefficient; instead, we fixed both $\\lambda_1$ and $\\lambda_2$ to 1.\n\n&nbsp;\n\nD. Preliminary Experimental Results\n\n|Dataset | Method                           | Accuracy (%) |\n|--------|-------------------------------------|-------------:|\n| **Books** (Source) |        -          |        97.21 |\n|**Electronics** (Target) |  **No adapt.**     |        **92.99** |\n| | **Tent** (EM)                  |        **87.73** |\n| | **+Data‑filter (entropy < 0.2)**  |        87.75 |\n| | **+SSM**                        |        92.31 |\n| | **+TCC**                        |        87.76 |\n| | **ReTTA** (EM+SSM+TCC)    |        **92.28** |\n\nThe DistilBERT model achieved 97.21% accuracy on the Books (source) domain. Without adaptation, the performance drops to 92.99% on Electronics (target). The results imply that the positive/negative sentiments from Books to Electronics highly overlap.\n\n* EM only (Tent) fell to 87.73%, showing that naive EM can be harmful under mild shifts.\n\n* Data filtering (entropy <0.2), adaptation with only those examples, yielded a marginal gain (87.75%) over EM.\n\n* EM with SSM recovered much of the gap, boosting adaptation accuracy to 92.31% on Electronics.\n\n* EM with TCC improved to 87.76%, which is marginally better than EM alone.\n\n* ReTTA (EM + SSM + TCC) achieved 92.28%, comparable to SSM only but markedly better than EM-only.\n\nThese findings suggest that when the target distribution is not too dispersed, such as in the case of Books (source) and Electronics (target), where distributions remain similar, naive EM tends to be limited or even counterproductive. In these scenarios, energy‑based optimization can offer a solution by enhancing local observability. While EM struggles to detect how the probability landscape shifts around individual test samples, lowering a sample’s energy via SSM pulls not only that sample but also nearby points in the space into lower‑energy (higher‑likelihood) regions. This broader effect may help fill the observable gaps that EM alone often misses and mitigates misclassifications. Therefore, when the distribution overlaps between source and target domains are significant, incorporating minimization of the energy may provide an opportunity to alleviate the penalties incurred by EM substantially.\n\n&nbsp;\n\nThis feedback helped us uncover that EM performs well under large shifts (e.g., ImageNet‑C) but underperforms when source and target domains are highly overlapped (e.g., text sentiment). This study allowed us to rethink entropy and energy again: by lowering energy, we can both improve adaptation performance and compensate for EM’s blind spots in overlap scenarios.\n\n&nbsp;\n\n> Minor comments\n\nThank you for your careful reading and helpful suggestions. We will add a description for Fig. 1d in its caption (refer to the caption of Fig. 2 in the Appendix for an example), bold the highest DeYO values in the accuracy tables, and clearly distinguish entropy minimization from energy minimization throughout the manuscript to improve clarity.\n\n&nbsp;\n\n---\n[1] Sanh et al., DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter, NeurIPS Workshop, 2019"}, {"review_id": "KDu1XWfz5P", "reviewer": "Reviewer_oKyZ", "summary": "Many existing test-time adaptation (TTA) methods are based on entropy minimization (EM), which encourages confident predictions on unlabeled test data. This paper revisits EM through its duality with energy, and argues that minimizing entropy alone is insufficient for TTA. The authors propose ReTTA, a unified framework that supplements EM with two additional objectives: (1) a sampling-free energy minimization objective via Sliced Score Matching (SSM), and (2) a Targeted Class Convergence (TCC) loss to promote discriminative predictions. Experiments on ImageNet-C demonstrate the effectiveness of ReTTA over existing EM-based and energy-based baselines.", "questions": "1. Figure 1 is not very clear to me, where I can't see much difference between Figure 1 (b) and (c). Are the authors suggesting that ReTTA (c) provides a clear improvement over EM (b)? If so, it would be helpful to highlight or quantify this difference more explicitly. Additionally, for (d), it would improve clarity to include a legend or annotation explaining the meanings of K and Z, so the figure is more self-contained.\n2. The paper claims ReTTA is a “scalable solution” to distribution shifts in Conclusion. What exactly does “scalable” mean here, in terms of data size, model size, or corruption severity? Is there any empirical evidence supporting this?\n3. I have doubt on the impact of TCC loss. Since test-time data is corrupted, the top-1 prediction may be wrong in some cases. Using it as a pseudo-target could mislead the model, especially in low-confidence cases. Table 3 suggests that TCC offers limited benefit when used alone. Could you provide more analysis on when and why TCC is effective?", "limitations": "The authors mentioned that the following in the checklist regarding the Limitations:\n\nAnswer: [Yes]. Justification: Limitations are discussed in the Appendix.\n\nBut it seems that I cannot access to the appendix...\nI suggest the authors add this part.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- Strengths\n1. The motivation of the proposed ReTTA method is welled analyzed. The insights of using both entropy and energy is valuable and novel.  \n2. ReTTA’s components (SSM and TCC) are easily pluggable into existing EM-based TTA methods, as demonstrated in the ablation studies (Table 4).\n\n- Weaknesses\n1. In the standard TTA setting (Table 1), ReTTA’s improvements over strong baselines like DeYO are relatively small (+0.6% average). The empirical results does not show a very significant advantage of using ReTTA, especially on mild scenario. \n2. While the method is motivated by covariate shift, the experimental setup includes a label shift, making it unclear what shift assumptions ReTTA is designed for. Given the shift in both p(x) and p(y), it is hard to say if this is still covariate shift or label shift since it's hard to maintain p(y|x) or p(x|y) unchanged.", "was_revised": true, "final_justification": "After reading the rebuttal and other reviews, all my concerns have been satisfactorily resolved. The authors further provide much additional experiments in the rebuttal to better empirically support the claim. So I have increased my score to vote for the acceptance.", "rebuttal": "We appreciate the valuable time and effort you took to provide constructive and positive feedback. We believe these clarifications strengthen our manuscript and look forward to favorable consideration.\n\n&nbsp;\n\n>W1. In the standard TTA setting (Table 1), ReTTA’s improvements over strong baselines like DeYO are relatively small (+0.6% average). The empirical results does not show a very significant advantage of using ReTTA, especially on mild scenario.\n>>Q1. Figure 1 is not very clear to me, where I can't see much difference between Figure 1 (b) and (c). Are the authors suggesting that ReTTA (c) provides a clear improvement over EM (b)? If so, it would be helpful to highlight or quantify this difference more explicitly. Additionally, for (d), it would improve clarity to include a legend or annotation explaining the meanings of K and Z, so the figure is more self-contained.\n\nThank you for your careful reading and observations. It is worth noting that even on mild shifts, leading TTA methods yield only modest gains. For instance, DeYO improves over EATA by +0.8% on average (Table 1), so ReTTA’s +0.6% over DeYO is almost in line with the state of the art. When applied to SAR, ReTTA yields a +2.7% net gain in accuracy (Table 4), corresponding to roughly 1,350 additional correct predictions out of 50,000. Although adaptation does flip a few accurate predictions to incorrect and vice versa, the dominant effect is this net gain of 1,350 samples. This improvement explains why Figs. 1(b) and 1(c) appear visually subtle. To clarify, we will annotate the figure with the number of samples corrected and add a legend for Fig. 1(d) (refer to the caption of Fig. 2 in the Appendix for an example).\n\n&nbsp;\n                                                                                   \n>W2. While the method is motivated by covariate shift, the experimental setup includes a label shift, making it unclear what shift assumptions ReTTA is designed for. Given the shift in both p(x) and p(y), it is hard to say if this is still covariate shift or label shift since it's hard to maintain p(y|x) or p(x|y) unchanged.\n\nAlthough the experimental protocols include both mild (covariate) and label shifts, this diversity cannot alter ReTTA’s core motivation: adapting to covariate shift in $p(x)$. In the mild setting, corruptions modify $p(x)$ under a uniform label distribution $p(y)$, so the resulting shift in $p(x|y)$ remains interpretable as a covariate shift. In label-shift tests, we replace the uniform prior with a skewed distribution $q(y)$, yielding $q(x)=\\sum_y p(x|y)q(y)$, which again manifests as a new covariate shift in the marginal $q(x)$. Thus, regardless of stress-testing under combined shifts, ReTTA’s design and motivation remain focused on adaptation for covariate shifts.\n\n&nbsp;\n\n>Q2. The paper claims ReTTA is a “scalable solution” to distribution shifts in Conclusion. What exactly does “scalable” mean here, in terms of data size, model size, or corruption severity? Is there any empirical evidence supporting this?\n\nWe intended to highlight ReTTA’s ability to adapt to diverse distribution shifts, made possible by its self-adjusting coefficient, since this mechanism enables the method to balance likelihood and entropy signals on the fly without the need for manual tuning across different distribution shifts. Now, in response to Reviewer GM3k, YKNW, and 34Nx, we have further evaluated ReTTA across ImageNet-C severities (1 and 3), various Out-of-Distribution domains (ImageNet-R, V2, S), and even text classification tasks. We believe that these results can also support our argument that ReTTA scales across different domain types, modalities, and shift intensities.\n\n&nbsp;\n\n>Q3. I have doubt on the impact of TCC loss. Since test-time data is corrupted, the top-1 prediction may be wrong in some cases. Using it as a pseudo-target could mislead the model, especially in low-confidence cases. Table 3 suggests that TCC offers limited benefit when used alone. Could you provide more analysis on when and why TCC is effective?\n\nThank you for raising this important point. Our motivation behind TCC is to amplify the effect of entropy minimization (EM) by explicitly targeting samples near truly zero entropy where correctness is more likely, as shown in our empirical observations. TCC offers a straightforward and effective method for pushing predictions into this regime. However, we also recognize that such aggressive guidance may break the conservative nature of EM and could be vulnerable to errors if applied indiscriminately.\n\nTo mitigate this, as shown in Algorithms 1–3, we employ strong entropy-based data filtering —a strategy widely used in state-of-the-art EM approaches —to ensure that TCC is applied only when confidence is sufficiently high. In this design, TCC is not intended to function alone, but as a complementary module within EM pipelines that already incorporate safety mechanisms.\n\nThus, Table 3 should not be interpreted as showing TCC’s limited effect in isolation, but rather as evidence of its effectiveness under proper safeguards. In fact, when combined with EM, TCC delivers dramatic improvements of +7% and +20% on corruptions like Glass and Frost, and an average gain of over +1% compared to the EM method."}, {"review_id": "Nrw2BS76Qj", "reviewer": "Reviewer_34Nx", "summary": "This paper proposes a new TTA method, called ReTTA, which balances out the widely used Entropy Minimization (EM) loss with an energy objective (Sliced Score Matching, SSM). Because entropy minimization only cares about the ratio of logits to each other, there is no incentive to increase the likelihood (lower the energy) of test inputs  In addition, Targeted Cross Entropy (TCC) is used to ensure that logits hit the 0 entropy mark, instead of hovering near it. ReTTA achieves state of the art results on ImageNet-C.", "questions": "Would it be possible to get more intuitive/practical explanations for the ways that the energy and TCC objectives help complement entropy? Are there specific images that are difficult for entropy based TTA to classify, but not for ReTTA?\n\nWould it be possible to see ReTTA performance on other ImageNet-size OOD datasets?\n\nWould it be possible to see ReTTA performance in the lifelong setting seen in EATA?", "limitations": "No, but I don't feel like this is relevant here.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper overall is well-structured and clear. There is theoretical motivation supplemented by practical results.\n\nStrengths:\n\nReTTA seems to significantly improve performance in the classic ImageNet-C setting. The results span architectures and can be stacked with other methods, namely EATA, SAR and DeYO.\n\nWeaknesses:\n\nI think more motivation for why the entropy minimization objective is not enough, and why the energy objective solves this. I can see the theoretical argument, yes, but I think having some sort of practical intuition would be very helpful. Figure 2 in [1] comes to mind; I think something along those lines, that shows the problem of minimizing entropy without minimizing energy would go a long way. In the same vein, getting some insight/intuition as to why TCC is helpful would also be interesting; when entropy is close to 0, I would imagine that the entropy loss and TCC are basically the same; if so, what does TCC do that entropy doesn't?\n\nI think the experiments should be more comprehensive; ImageNet-C Severity 5 is good enough to get a good understanding of the method's performance, but full IN-C, or at least, levels 1, 3, along with 5 should be shown. I also think ImageNet-R, ImageNet-3DCC, ImageNet-V2 or other popular, ImageNet-scale OOD datasets could boost the robustness of the results table. \n\nHow effective is ReTTA in the lifelong setting seen in [1] (Table 2)? EATA showed that forgetting was a problem with many top TTA methods, it would be interesting to see how ReTTA handles it. \n\n\n\nMinor: \nI think it would be helpful to have an explanation of the settings \"Label Shift\" and \"Mild\" in the paper, even if they are identical to ones mentioned in other papers. \nFigure 4a shows that $\\lambda_2$ was chosen in based on the same experimental settings seen in Table 1. If I understand this correctly, this should not be done; parameters should be picked using the IN-C holdout set of noises, not the 15 main noises. \nTable 3, the rows in the first columns are all the same (should be EATA, SAR, DeYO)\n\n[1] Niu, Shuaicheng, et al. \"Efficient test-time model adaptation without forgetting.\" International conference on machine learning. PMLR, 2022.", "was_revised": true, "final_justification": "I found the paper to be easy to read, with a novel method that made sense. The paper provided only a select number of benchmarks at first, but with the rebuttal, the authors have demonstrated ReTTA's effectiveness across: IN-C severities 1 and 3, the continual setting used in EATA, as well as other OOD ImageNet scale datasets. My concerns have been addressed, and I have no further concerns.", "rebuttal": "We appreciate the valuable time and effort you took to provide constructive and positive feedback. We believe these clarifications strengthen our manuscript and look forward to favorable consideration.\n\n&nbsp;\n\n>W1&Q1. When entropy is close to 0, I would imagine that the entropy loss and TCC are the same; if so, what does TCC do that entropy doesn't? Would it be possible to get more intuitive/practical explanations for the ways that the energy and TCC objectives help complement entropy? Are there images that are difficult for entropy based TTA to classify, but not for ReTTA?\n\nWe view TCC as a simple yet effective way to push the model output from merely low entropy to truly zero entropy, something that EM alone cannot ensure. Intuitively, the motivation is straightforward since zero-entropy predictions often correspond to highly confident and accurate classifications, as shown in Figs. 1(b) and 1(c). When a prediction is already at zero entropy, neither EM nor TCC makes further updates. In such cases, SSM pulls the sample deeper into the high-density, in-distribution region that EM alone cannot reach. By doing so, we encourage adaptation toward in-distribution and likely correct samples. So, for most inputs with non-zero entropies, TCC remains essential: in practice, as Table 3 shows, coupling state-of-the-art EM with TCC yields dramatic gains of 7% or 20% on corruptions like Glass or Frost, and achieves over a 1% average improvement versus the EM method (our in-house test of TCC with Tent shows a 0.9% average gain, from 43.1% to 44.0%). We also find EM often fails on images where the object is tiny or occluded, whereas ReTTA succeeds. A comprehensive analysis will require new metrics, which we plan to analyze in-depth.\n\n&nbsp;\n\n>W2. full IN-C, or at least, levels 1, 3, along with 5 should be shown.\n\nWe observed that at severity level 1, all TTA methods achieved comparable average performance (with ReTTA only marginally superior), whereas as corruption severity increased (levels 3 and 5; see Table 1), the combination of energy-based and entropy-based optimizations became important for maintaining higher accuracy.\n\nA. ImageNet-C Severity 1 (ResNet-50-BN)\n\n| Method         | Gauss. | Shot  | Impul. | Defoc. | Glass | Motion | Zoom  | Snow  | Frost | Fog   | Brit. | Contr. | Elastic | Pixel | JPEG  | Avg   |\n|----------------|-------:|------:|-------:|-------:|------:|-------:|------:|------:|------:|------:|------:|-------:|--------:|------:|------:|------:|\n| No adapt       |   59.6 |  57.8 |   48.4 |   59.0 |  53.7 |   64.5 |  52.2 |  54.3 |  61.1 |  61.4 |  73.8 |   64.4 |    66.4 |  63.9 |  65.9 | 60.4 |\n| Tent           |   68.0 |  67.6 |   64.3 |   66.4 |  67.3 |   70.5 |  65.4 |  65.1 |  66.4 |  70.2 |  **74.6** |   72.0 |    69.9 |  72.0 |  70.1 | 68.7 |\n| EATA           |   68.7 |  68.4 |   65.6 |   67.0 |  68.0 |   70.6 |  66.2 |  66.5 |  66.9 |  70.8 |  74.4 |   72.3 |    70.2 |  72.3 |  70.4 | 69.2 |\n| SAR            |   67.9 |  67.5 |   64.3 |   66.3 |  67.3 |   70.4 |  65.4 |  64.9 |  66.3 |  70.1 |  74.5 |   71.9 |    69.9 |  72.0 |  70.1 | 68.6 |\n| DeYO           |   68.9 |  68.6 |   66.0 |   67.2 |  68.0 |   70.5 |  66.3 |  **67.0** |  67.1 |  70.8 |  74.1 |   72.2 |    70.1 |  **72.3** |  70.2 | 69.3 |\n| TEA            |   59.4 |  58.9 |   54.0 |   56.8 |  58.5 |   61.6 |  54.7 |  56.9 |  57.0 |  62.4 |  66.8 |   63.6 |    60.3 |  64.0 |  61.2 | 59.7 |\n| **ReTTA**      |   **68.9** |  **68.7** |   **66.0** |   **67.4** |  **68.3** |   **70.7** |  **66.3** |  66.9 |  **67.1** |  **70.8** |  74.1 |   **72.4** |    **70.3** |  72.2 |  **70.4** |**69.4** |\n\nB. ImageNet-C Severity 3 (ResNet-50-BN)\n\n| Method         | Gauss. | Shot  | Impul. | Defoc. | Glass | Motion | Zoom  | Snow  | Frost | Fog   | Brit. | Contr. | Elastic | Pixel | JPEG  | Avg   |\n|----------------|-------:|------:|-------:|-------:|------:|-------:|------:|------:|------:|------:|------:|-------:|--------:|------:|------:|------:|\n| No adapt       |   27.6 | 25.0  |   25.2 |   37.9 |  16.9 |   37.7 |  35.2 |  35.2 |  32.1 |  46.7 |  69.6 |   46.0 |    55.6 |  46.2 |  59.3 | 39.7 |\n| Tent           |   54.8 | 54.3  |   53.7 |   49.2 |  46.5 |   58.8 |  57.7 |  55.9 |  48.6 |  65.8 |  72.1 |   67.1 |    69.4 |  67.5 |  65.9 | 59.2 |\n| EATA           |   57.0 | 56.8  |   56.2 |   52.4 |  50.2 |   61.0 |  59.7 |  58.6 |  51.3 |  67.1 |  72.2 |   68.2 |    69.9 |  68.1 |  66.7 | 61.0 |\n| SAR            |   54.6 | 54.1  |   53.5 |   49.3 |  46.3 |   58.6 |  57.6 |  55.6 |  48.6 |  65.6 |  72.0 |   67.1 |    69.3 |  67.3 |  65.7 | 59.0 |\n| DeYO           |   58.1 | 58.0  |   57.1 |   53.2 |  51.2 |   61.9 |  59.8 |  59.6 |  51.9 |  67.5 |  72.0 |   68.5 |    69.6 |  68.6 |  66.6 | 61.6 |\n| TEA            |   24.8 | 22.0  |   24.5 |   20.1 |  14.9 |   45.3 |  32.0 |  35.0 |  17.6 |  57.8 |  64.2 |   50.9 |    60.4 |  59.0 |  55.5 | 38.9 |\n| **ReTTA**      |   **58.6** | **58.7**  |  **58.0** |   **53.2** |  **51.5** |   **61.9** |  **59.9** |  **59.7** |  **52.1** |  **67.5** |  **72.3** |   **68.7** |   **69.9** |  **68.8** |  **66.9** |**61.9** |\n&nbsp;\n\n>W2&Q2. Would it be possible to see ReTTA performance on other ImageNet-size OOD datasets?\n\nWe evaluated ReTTA under `mild’ settings on three additional ImageNet-scale OOD benchmarks. ImageNet-R (rendered images) exhibits a large domain gap compared to ImageNet, while ImageNetV2 overlaps the original ImageNet distribution, and ImageNet-S (sketches) consists of single-channel grayscale outlines. We were unable to include ImageNet-3DCC, as its download alone exceeded 3 days and more than 340 GB and stalled. Therefore, we substituted ImageNet-S instead.\n\nA. ImageNet R\n\n| Method         | ResNet 50 GN | ViT Base LN |\n|----------------|-------------:|------------:|\n| No Adapt.      |         40.8 |        50.9 |\n| Tent           |         42.8 |        55.3 |\n| EATA           |         41.9 |        51.1 |\n| SAR            |         41.9 |        51.8 |\n| DeYO           |         47.0 |        61.3 |\n| TEA            |          7.0 |        22.9 |\n| **ReTTA based on EATA**    |         42.4 |   52.1 |\n| **ReTTA based on SAR**     |        42.3 |    52.5 |\n| **ReTTA based on DeYO**      |    **47.4** |    **61.7** |\n\nB. ImageNetV2\n\n| Method         | ResNet 50 BN |\n|----------------|-------------:|\n| No Adapt.      |        **63.20** |\n| Tent           |        63.07 |\n| EATA           |        63.14 |\n| SAR            |        63.06 |\n| DeYO           |        62.89 |\n| TEA            |        57.28 |\n| **ReTTA based on EATA**    |        **63.20** |\n| **ReTTA based on SAR**     |        63.14 |\n| **ReTTA based on DeYO**    |        63.06 |\n\nC. ImageNet S\n\n| Method         | ResNet 50 BN |\n|----------------|-------------:|\n| No Adapt.      |        24.09 |\n| Tent           |        30.50 |\n| EATA           |        35.24 |\n| SAR            |        31.74 |\n| DeYO           |        35.83 |\n| TEA            |         8.87 |\n| **ReTTA based on EATA**    |        35.24 |\n| **ReTTA based on SAR**     |        32.16 |\n| **ReTTA based on DeYO**    |        **35.86** |\n\nOn ImageNet-R, ReTTA achieves gains comparable to those on ImageNet-C. On ImageNetV2, where other methods sometimes experience a performance drop, ReTTA offsets that penalty, paralleling our findings in the text-classification experiments (Response to YKNW). On ImageNet-S, ReTTA maintains the top rank among baselines, although the limited grayscale information and then reduced modes of distribution likely constrain the impact of SSM.\n\n&nbsp;\n\n>W3&Q3. Would it be possible to see ReTTA performance in the lifelong setting seen in EATA?\n\nLooking back at Lemma 3, we could see that SSM might also help prevent forgetting. In each update, the “negative phase” gently raises energy (reducing the likelihood) for samples from the source distribution, while the “positive phase” lowers energy on the new corrupted samples. Rather than canceling out in a single SGLD iteration, these forces may reach a balance that slightly broadens the source density to cover new data without erasing its original support. \n\nIndeed, when comparing ReTTA + EATA in the standard TTA setting (Table 4), we observe a modest average gain of +0.2%, and in the lifelong protocol, an even larger boost of +0.3%. These results suggest that SSM and EATA can be complementary not only for adapting to distributional shifts but also for preserving source performance over time, as evidenced by the average accuracies of 74.0% and 73.9%, respectively.\n\n\n| Method                         | Gauss. | Shot  | Impul. | Defoc. | Glass | Motion | Zoom  | Snow  | Frost | Fog   | Brit. | Contr. | Elastic | Pixel | JPEG  | Avg   |\n|--------------------------------|-------:|------:|-------:|-------:|------:|-------:|------:|------:|------:|------:|------:|-------:|--------:|------:|------:|------:|\n| **EATA (Continual)**           |   35.3 |  38.7 |   38.0 |   34.1 |  **34.0** |   47.1 |  52.9 |  50.9 |  **45.6** |  59.8 |  67.9 |   44.1 |    57.4 |  60.1 |  54.9 | 48.0 |\n| **Source**          |   73.8 |  73.5 |   73.6 |   73.7 |  73.5 |   73.8 |  74.1 |  74.3 |  74.1 |  74.6 |  74.9 |   74.1 |    74.2 |  74.3 |  74.0 | **74.0** |\n| **ReTTA + EATA (Continual)**   |   **36.0** |  **38.8** |  **38.2** |   **34.2** |  33.8 |  **47.4** |  **53.1** |  **51.4** |  45.5 |  **59.9** |  **68.1** |  **44.6** |  **57.5** |  **60.7** |  **54.9** | **48.3** |\n| **Source**          |   73.4 |  73.5 |   73.3 |   73.6 |  73.4 |   74.0 |  74.1 |  74.1 |  74.0 |  74.4 |  74.7 |   74.0 |    74.1 |  74.4 |  73.9 | 73.9 |\n\n&nbsp;\n\n>Minor weaknesses\n\nThank you for your careful reading and helpful suggestions. To address your remaining concerns, we will (1) add brief descriptions of the “Label Shift” and “Mild” settings; (2) due to the volume of experiments, we will reserve a small holdout subset of ImageNet-C corruptions for exploring $\\lambda_2$, and (3) update Table 3 to include the missing EATA and SAR entries."}]

Author rebuttal:

Meta-review: This work introduces ReTTA, a novel test-time adaptation method that goes beyond standard entropy minimization to address the likelihood gap under covariate shifts. By combining enhanced entropy minimization, energy-based adaptation, and a self-balancing mechanism, ReTTA achieves more reliable and effective adaptation, marking a distinct contribution to the field.

Initially, the reviewers raised several concerns regarding runtime efficiency, hyper-parameter sensitivity, effectiveness across diverse datasets, models, and tasks (including text and speech modalities), as well as broader scenarios such as lifelong adaptation and evaluations under mild shifts. They also requested clearer motivation and methodological details. During the rebuttal, the authors provided thorough discussions and additional empirical evidence, which substantially improved the paper’s quality and convincingly addressed concerns from all reviewers.

Overall, both the reviewers and the AC recognize the proposed method as simple yet effective, offering a new fully test-time adaptation framework with broad applicability. The motivation is theoretically grounded, and the empirical validation has been strengthened with extensive new results after the rebuttal. Accordingly, the AC recommends this work for a “spotlight” presentation.

Please ensure that the new rebuttal results and discussions are incorporated into the final version.

ps. The AC also encourages the authors to further explore and compare performance in other wild settings (e.g., batch size of 1 and mixed domain shifts). Such analyses would not only strengthen the justification of the method’s superiority but also offer valuable insights and comparative benchmarks for future work. The inclusion of these results and discussions in the final version is recommended.
Decision: Accept (spotlight)