Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approach
Authors: Yarin Bar
Shalev Shaer
Yaniv Romano
Venue: NeurIPS 2024 poster (2024)
Keywords: Test Time Domain Adaptation
Online Learning
Testing by Betting
Martingale
Distribution Shift Detection
Primary area: safety_in_machine_learning
TL;DR: A novel self-training approach for adapting ML models to test-time distribution shifts by monitoring the model's output and aligning it with the source domain's statistics.

Abstract: We present a novel approach for test-time adaptation via online self-training, consisting of two components. First, we introduce a statistical framework that detects distribution shifts in the classifier's entropy values obtained on a stream of unlabeled samples. Second, we devise an online adaptation mechanism that utilizes the evidence of distribution shifts captured by the detection tool to dynamically update the classifier's parameters. The resulting adaptation process drives the distribution of test entropy values obtained from the self-trained classifier to match those of the source domain, building invariance to distribution shifts. This approach departs from the conventional self-training method, which focuses on minimizing the classifier's entropy. Our approach combines concepts in betting martingales and online learning to form a detection tool capable of quickly reacting to distribution shifts. We then reveal a tight relation between our adaptation scheme and optimal transport, which forms the basis of our novel self-supervised loss. Experimental results demonstrate that our approach improves test-time accuracy under distribution shifts while maintaining accuracy and calibration in their absence, outperforming leading entropy minimization methods across various scenarios.

Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approach
Yarin Barl*
Shalev Shaer
Yaniv Romano 1.2
'Department of Computer Science, Technion Israel Institute of Technology
2 Department of Electrical and Computer Engineering. Technion- Israel Institute of Technology Cyarinbar, shalev. shaer Acampus technion aaaaiion.ac...... ac
yromano@technior ac.
Abstract
We present a novel approach for test-time adaptation via online self- trainingn con- sisting of two components. First, we introduce statistical framework that detects distribution shifts in the classifier's 's entropy values obtained on a stream of unla- beled samples. Second, we devise online adaptation mechanism that utilizes the evidence of distribution shifts captured by the detection tool to dynamically update the classifier's parameters. The resulting adaptation process drives the distribution of test entropy values obtained from the self-trained classifier to match those of the source domain, building invariance to distribution shifts. This approach departs from the conventional self-training method, which focuses on minimizing the classifier's ' entropy. Our approach combines concepts in betting martingales and online learning to form a detection tool capable of quickly reacting to distribution shifts. We then reveal tight relation between our adaptation scheme and optimal transport, which forms the basis of our novel self-supervised loss. Experimental re- sults demonstrate that our approach improves test-time accuracy under distribution shifts while maintaining accuracy and calibration in their absence, outperforming leading entropy minimization methods across various scenarios.
Introduction
The deployment of machine learning (ML) models in real-world settings presents significant challenge, as these models often encounter testing environments (target domains) that differ from their training, source domain 1--15. Consider, for example, an image recognition system employed for medical diagnostic support (6] where the quality of images acquired during testing deviates from the training data due to factors such as equipment degradation and novel illumination conditions. ML models are sensitive to such distribution shifts, often resulting in performance deterioration, which can be unexpected [15]. Ultimately, we want predictive models to dynamically adapt to new testing environments without the laborious work required to annotate new, up-to-date labels.
Recognizing this pressing need, there has been surge in the development of adaptation methodologies to enhance test-time robustness to shifting distributions [21H28]. One commonly used approach involves jointly training model both the source and target domains 24 However, such train-time adaptation methods assume access to unlabeled test data from the target domains, limiting the ability to adapt the model to new domains that emerge during testing. To overcome this limitation, test-time adaptation techniques offer strategies that dynamically update the model parameters as new unlabeled test points become available. particular, leading methodologies draw inspiration from
Equal Contribution.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

the relationship between the entropy of estimated class probabilities measure of confidence and model accuracy [3544] Empirical evidence highlights that lower entropy often corresponds to higher accuracy, encouraging the development of self-supervised learning approaches that adjust model parameters by minimizing the entropy loss or the cross-entropy through the assignment of pseudo- or soft-labels to test points 44]
While test-time adaptation techniques have shown promise in enhancing test accuracy under domain shifts, there is a caveat: minimizing entropy or related self-supervised loss functions without control can lead to overconfident predictions, and may suffer from undesired, noisy model updates [49 [2]. In extreme cases, this approach may even cause the model to collapse and produce trivial predictions [53] [54]. Indeed, without careful implementation and tuning. these techniques may not improve- or could even reduce [4] - predictive performance in realistic settings.
In this paper, we present novel, statistically principled approach to test-time adaptation via self- training. Our methodology is built upon two key pillars. First, we introduce an online statistical framework that monitors and detects distribution shifts in the test data influencing the models' predictions. We achieve this by sequentially testing whether the distribution of the classifier's entropy values obtained during testing deviates from the ones corresponding to the source domain. Armed with this monitoring tool, we then devise an online adaptation mechanism that leverages the accumulated evidence of distribution shifts adaptively update model parameters. This mechanism drives the distribution of the self-trained classifier's 's entropy values, obtained on test data, to closely match the distribution of entropy values when applying the original model the source domain. As result, our proposed Protected Online Entropy Matching (POEM) method adapts the model on the fly in controlled manner: the absence of a distribution shift, our approach has "no-harm" effect both on accuracy and calibration of the model, whereas under distribution shifts our experiments demonstrate an improvement of the test time accuracy, often surpassing state-of-the-ar methods.
Contributions
(i) We present a sequential test for classification entropy drift detection, building on betting martin- gales (55457) and online learning optimization (58)60) to provably attain fast reactions to shifting data. (ii) Inspired by [61], we show how to utilize the test martingale to analytically design a mapping function that transports the classifier entropies obtained at test time to resemble those of the source domain. Under certain assumptions, we establish connections between our online testing approach and optimal transport [62] as a mechanism for distribution matching. (iii) This observation sets the foundation of the entropy-matching loss function used in POEM. (iv) Numerical experiments in both continual and single-shift settings demonstrate that our approach is competitive and often outperforms strong benchmark methods that build on entropy minimization. These experiments are conducted using commonly used predictive models (ViT [63] and ResNet (64) on standard
benchmark datasets: ImageNet-C [65]. CIFARIO-C, CIFAR100-C, and OfficeHomme [66]. A software
package that implements our methods is available httph::///eee tit.....coooo. wtw.tt...o/aaaribrrr/aaaoae
2 Preliminaries
2.1 Problem setup
To formalize the problem, consider a K-class classification problem with labeled training data (Xg, ( from a source domain, sampled i.i.d. from the source distribution Pxy- Here, X* E Rdd represents observed covariates and Y* E (1....,4$ } is the corresponding label. During testing. we encounter a stream of points X; with unknown labels Y sampled from an unknown target
K}
distribution P that may shift over time j 1,2,.... To define the shifting mechanism, let T3 : Idd >> Rdd be an unknown corruption/shift function, resulting in test instances X₁- (X*) with X" being a fresh sample from p For instance, in datasets such ImageNet-C, corruptions involve modifications such as blur or changes in illumination applied to clean source images. While such transformations alter the marginal P distribution of the target domain, we assume that the conditional distributions of Ps p (Y" X*) and P/ (X**) are the same for all j = 1, 2.....
[67].

---

2.2 Related work: fest-time adaptation via self-training
Given pre-trained classifier fg trained on the source domain, leading test-time adaptation approaches build on the idea of self-training to update the model parameters sequentially. Denote the adapted classifier by fيو. where represents the modification to the parameters of the original model 0 obtained by self-training during testing. The adaptation process is typically initialized with j = 1 and w/ 0, and involves the following set of steps:
1. Observe fresh test point X3 and predict its unknown label using fâ+w, (X3).
2. Update the model parameters in a direction that reduces the self-supervised loss, i.e.
Wj+1 t Wj V (fote)
where the papee-paraeeter 7j is the step size.
3. Set t j + 1 and return to step 1.
A common choice for eself in test-time adaptation methods is the entropy loss:
K gent (Jāzw())) = Σ
(1) (og, (z))
(1)
y=1
where (x)y is the y-th entry of the classifier's 's softmax layer, we omit the index j of w for clarity.
While entropy minimization has been shown to enhance test-time robustness (3-43] it is also prone to instabilities [49]51 68] For instance, enforcing ( = 1 for a fixed entry y minimizes gom but it collapses the classifier to make trivial predictions. To alleviate this, various strategies have been proposed. For example, one approach is to avoid training on samples with high entropy [42], as high entropy often correlates with erroneous pseudo labels. Another example is the utilization of fallback mechanism that resets the adapted model back to the original model fg when the average entropy becomes too small [3). A more detailed review of test-time adaptation methods is given in Appendix AA] Importantly, this line of work underscores the limitations of entropy minimization and highlights the need to better control its effect. This aligns with the goal of our work, which offers a distinct, statistically grounded approach for test-time adaptation. While we also build on the classifier's s entropy to form the adaptation mechanism, we could integrate any alternative self-supervised loss in our online distributional matching scheme.
2.3 Testing by betting
A key component of our method is the proposal of an online test for distribution drift. The design of this test follows the framework of testing-by-belting (69). Intuitively, one can interpret this testing framework as participating
a fair game. We begin with initial toy money, and at each time step, we observe a new test point and place a bet against the null hypothesis we aim to test. If the bet turns out to be correct, our wealth increases by the money we risked in the bet: otherwise, we lose, and our wealth decreases accordingly. Mathematically. the wealth process is formulated as a non-negative martingale, where a successful betting scheme reflected in a growing martingale (wealth) trajectory, offering progressively stronger statistical evidence against the null hypothesis. However, if the null hypothesis is true, the game must be fair in the sense that it unlikely significantly grow our initial capital, no matter how sophisticated our betting strategy may be. This implies that, under the null hypothesis, unlikely that the martingale will grow significantly beyond its initial value.
The serinn-by--ettting framework is widely used in sequential settings. Notable examples include: one and two sample tests 77 independence and conditional independence tests 73), ex- changeability tests (56, 74H78) and more (69] (9)22). This framework is also used for change-point detection and testing for uniformity (57] 95- related our drift detection problem. We draw inspiration from the protected probabilistic regression approach [61] that combines the probability integral transform and betting martingales to improve the robustness of cumulative distribution function (CDF) estimator to distribution shift in the data. The experiments in [6]] illustrate this method's ability to enhance the accuracy of a regression model, where this protection scheme assumes access to new up-to-date labels. In contrast, we focus on a completely different setup where the labels of the test points are unknown, showing how the protected regression approach can be generalized to form a self-supervised loss function. In turn, we introduce two key contributions to test-time
3

---

4. Leverage the betting function to obtain an adapted pseudo-entre enturpy value Z, that better matches the distribution of Z*. The intuition here is that we derive 2, in a way that would reduce the toy money we would have gained if we had used the same betting strategy on Z,. adaptation via be-binngbb-eetingg..... First, we present an adaptive online learning technique to optimize the betting mechanism. Second, we pioneer the application of testing- -a-gg--b-beee by-bettibgggggg in this domain.
3 Proposed method: protected online entropy matching (POEM)
3.1 Preview of our method
Let the random variable z = [fa be the entropy value of the original classifier applied to fresh sample X from the source domain. We refer to this variable as the source entropy. In addition, denote by 2 = ( (f+ ))) by the updated model, evaluated on stream of unlabeled test data. We refer to Z' as the targer entropy. Our proposal uses the source and target entropies both to detect distribution shifts and adapt the model to new testing environments without relying on up-to-date labeled data. The rationale behind our method is as follows: when test data sampled from the source distribution, there will be no deviation between the source and target entropies, implying that there is no need to adapt the model. However, statistical deviations between the source entropies Z" and target entropies Z' can indicate that the model encounters test data different from the training distribution. This motivates us to introduce self-training framework that encourages the model to generate test-time entropies z that closely resemble the source entropies Z to build invariance shifting data.
1.2..... a sequence of entropy values generated
To achieve this goal, we utilize the testing-by by bettint approach and formulate our adaptation scheme as game, in which we start with initial toy money and proceed as follows.
1. Place bet against the null hypothesis that the unknown classifier entropy Zj of the upcoming test point X3 will follow the same distribution as the source entropies Z
2. Observe the test point X5. predict its label using the model fatwy, and compute the classi- fier's 's entropy 2 gem (fo+wy (Xf)).
3. Given Lj. reveal the outcome of the bet using a betting function. If the bet is successful, increase the accumulated wealth; otherwise, decrease it.
5. Update the model parameters: obtain Wi+1 by taking a gradient step that reduces the self-supervised matching loss
ematch (z), = (100 (XJ)) 2,)2
6. Update the betting strategy for the next round and return to Step 1.
In the following sections, we describe in detail each component of the proposed adaptation scheme. Before proceeding, however, we pause to highlight the advantages of the matching loss (2) over entropy minimization.
3.2 Motivating example: entropy minimization vs. entropy matching
To facilitate the exposition of the proposed loss, it is useful to consider a toy, binary classification example with a mie-dimensional input X in which we have oracle access both to the source Pxyy and a fixed target distribution P that does not vary over time. We commence by generating training data from pxy, where P(Y = 1) = P(Y 1) and p = >N 1). See Figure for an illustration of the source distribution. Throughout this experiment, we set the pre-trained Gaussian classifier fg to be the Bayes optimal one for which 0 = O, and during test-time we optimize the parameter w of the updated classifier. Since 0 = 0, in this case fo+ is simplified to fa. Further implementation details are provided in Appendix FF
As mentioned before, one of the advantages of our approach is its "no-harm" effect, i.e., when P = Pxy we ideally want to keep the decision boundary of the classifier intact. The red curve
In the experiments in Section 4 we variation of this loss, described in Section 3.5

---

Class -1
Class +1
No Adapt Acc 0.84
Risk
E(("*]
No-Adapt
Ent Matching 0.84
Ent Min. Acc 0.50
Adapt 0.74
Risk
Ent Min. Acc 0.50
Ent Matching Acc - 0.84
X (also w)
Figure 1: Demonstration of the advantage of entropy matching on toy binary classification problem with Gaussian data. The top panel represents an in-distribution setup in which P Pxy- The bottom panel illustrates an out- -itt-f--iisributionn setup, obtained by shifting the two Gaussians. The entropy matching (red) and entropy minimization (black) risks are presented as a function of w. The dashed yellow line presents the decision boundary of the pre-trained classifier. The points marked by stars correspond to the decision boundary of the adapted classifiers
in Figure illustrates the entropy matching risk Ex [ (z, 2)] as a function of the classifier parameter W. In this synthetic case, the ideal entropy matching risk can be evaluated since we have access to the generating distribution: we can obtain the ideal pseudo-entropy Z', given by 2 (P' fw),, where F. and F, are the CDF functions of the source and target entropy values, respectively; the formula above is nothing but the optimal transport map. Of course, in the practical online setting we consider in this work, F; is unknown and varies over time. In fact, this is also true in the case studied here as the distribution of Z' = fx*( varies with w, highlighting the importance of our online, adaptive testing procedure. Indeed, the w obtained by our online adaptation scheme (POEM) minimizes the entropy matching risk and remains close to O, as desired.
Meanwhile, the black curve in Figure iillustrate the values of the entropy risk Ex [8en (2') for varying w. In contrast with our approach, the that minimizes the entropy risk is far from the optimal classifier. This approach results in a collapse towards trivial classifier that always predicts - 11, regardless of the value of X Moving to an out-of-distribution scenario, where we consider test data sampled from shifted version of the two Gaussians such that Y = Y and X = x*+1. Following the bottom panel in Figure T is evident that by minimizing the proposed entropy matching risk, the accuracy of the original (not adapted) classifier is effectively restored. In contrast, the entropy minimization paradigm once again collapses the model to make trivial predictions.
3.3
Online drift detection
We now turn to introduce a rigorous monitoring tool that capable of detecting whether the distribu- tion of the adapted classifier's s entropy values Z', obtained at test time, deviate from 7"ethe source entropies obtained by applying the original model to the source data. To detect such shifts, we assume that we have access to Fs. the CDF of the source entropy values Z* ('x("))))) In practice, we estimate this CDF using holdout unlabeled samples X* from the source distribution. Armed with Fs, we then apply the probability integral transform (96). allowing us to convert any sequence of i.i.d. entropy values from the source distribution Zf: Zz.... into a sequence of i.i.d. uniform random variables F, (Z)), FZ()), ... the [0, 1] interval. Therefore, if we observe sequence F,( F,()), of i.d. uniform variables at test time, we can infer that the target entropy distribution matches the source entropy distribution. Thus, if the sequence of transformed variables (Z),), F. (.i)),.... deviates from the uniform distribution, we can infer that the corresponding target domain samples 7 77, ... differ from the source distribution. This observation lies the core of our monitoring tool [61].
Specifically, we leverage the testing- by--tttttnngtggg- approach to design sequential test for the following null hypothesis:
Ho : u, & F.(Z)) 10, Vj e N,
33)
5

---

where - - (X))). In words, we continuously monitor the sequence of random variables U1, u2.... and test whether these deviate from the uniform null. We do so by formulating a test martingale, defined as follows.
Definition 1 (Test Martingale). A random process {$y i E N, So = 1) is a test martingale for the null hypothesis Ho if it satisfies the following:
1. S 2 VjE E.
2. {S, j E No} is a martingale under Ho, i.e. Es S1,....,j-1]] Sj-1]=
The martingale can be thought of the wealth process in the game-theoretical interpretation of the test, obtained by betting toy money against Ho as new data points arrive. We initialize this game with So = and update the wealth process as follows [6]]
Sy(u & Sy-1 b(u,) where b(u,) = 1111+,00 (u, (..).
(4)
Above, b(u) E [0, 2] is the betting function. The betting variable e, E 2- controls how aggressive the bet is, and it can be determined based on past observations "1...... -uj-1, as we detail later in this section. However, before introducing our strategy to update E, over time, we should first discuss the properties of the betting function b(u,). The idea behind this choice is that we sequentially test whether the sequence of U1. ., observed up to time step j has mean 0.5. Indeed, if the null is true, lE [x] 0.5 and thus EHa [b(u] 1. As result, under the null, the martingale is unlikely to grow significantly beyond its initial value-this is consequence of Ville's 's inequality; can gather evidence against the uniform null hypothesis by placing more aggressive bets, especially when past observed values u1,.... "3-1 deviate significantly from 0.5. This highlights the role of e,, controlling the value and direction of our wagers in each round, betting on whether uj would be over/under 0.5. Following 4 when ej and (u, - 0.5) have the same sign, we win the bet and obtain b(u,) 1. This implies that our capital increase as Sy,5 := Sy-1 b(u,). Notice that, in this case, a larger Ej (in absolute value) will allow us increase the capital more rapidly, resulting in a more powerful test. However, e and (u, - 0.5) have different signs, we lose the bet and obtain b(u,) < 1. Here, a larger e, would incur a more significant loss of capital. The challenge in choosing 6j lies in the restriction that cj can only be determined based on past experience, i.e., we must set its value without looking at the new uy. This restriction is crucial to ensure the validity of the martingale, as detailed the following proposition.
see Appendix D However, if the null is false,
Proposition 1. The random process presented in 44 is a valid test martingale for Ho (3)
The proof is given in Appendix C.I it is well-known result, see, e.g., 161. This property is crucial to form the proposed online distributional matching mechanism, introduced in the next section. Appendix D provides further details on how the test martingale is used for distribution shift detection.
We now turn to present an adaptive approach to set the betting variable ej in manner that enables powerful detection of drifting target entropies. This is especially important given the dynamic nature of both the target data and the continuous, online updates of the model. To achieve this, we adopt an online learning technique to learn 6j from past observations, with the goal of maximizing the wealth by minimizing the negative log of the wealth process up step j [70]:
)
- log((S((,)) -log II by (u,)= 7=1
= - log(b, by (u+)) Σ log(1 + e, Ex - 0.5)). (5)
7=1
This formulation allows us to learn how to predict e, using past samples via gradient descent 70
Specifically, our optimization approach relies on the scale-free online gradient descent (SF-OGD) algorithm [59]. Importantly, extending SF-OGD to our setting is not straightforward, since e must be in the range of 2] to form a valid test martingale. In the interest of space, we present this algorithm and its theoretical analysis in Appendix B and only highlight here its key feature. SF-OGD allows us to attain an anytime regret guarantee, which is presented formally in Theorem D of the Appendix. This guarantee bounds the difference between the negative log of the wealth process (i) obtained by the predicted 6, over time horizon 1 < 1 I j. and (ii) obtained by the best betting variable c* that can only be calculated in hindsight, after looking at the data up to time < t j- Informally, our theory shows this regret is bounded by c etV for all < S j. where the constant e depends on the problem parameters; it is formulated precisely in Theorem [ In turn, the anytime regret guarantee confirms that our SF-OGD approach effectively learns éj. capturing the dynamic changes of both the target distribution and the model Sāaw, in a fully online setting.

---

3.4 Online model adaptation
Having established a powerful betting strategy, we now turn to show how transform the test martingale {Sy : j E N into a sequence of adapted pseudo-entropy values Z1, Zz,... ... that better match the distribution of Z In what follows, we describe an algorithm to obtain Zj. which draws inspiration from [6]], and then connect this procedure to optimal transport.
Our adaptation scheme leverages the fact that any valid betting function is essentially likelihood ratio 6 82 This property implies that our betting function b(u,) = 1 + t,(, - 0.5) satisfies
dQ(ug) b(u,) aG(ug))
(6)
where dQyu)) and dG(u,) are the densities of some alternative distribution Q and the null distribution G. respectively. In our case, the null distribution is the uniform distribution Uniform (0, 1), and the alternative distribution Q can be intuitively thought of an approximation of the unknown target entropy's CDF; we formalize this intuition hereafter. Leveraging this likelihond ratio interpretation, we follow [6] and extract the alternative distribution Q by re-writing (6) as dQ(ug) = (Mus))dG(( and computing the integral:
((u))= b( b(o)dG(o)do = S b(u) 1 do 20 23 + (1 ) ug.
(7)
Above, we used the fact that the null density is dG(vv equals 1 over the support [0, 1]. Having access to Q. we can compute the adapted ü, =- Quu,) that can be intuitively interpreted as the result of applying the probability integral transform to 2j using the estimated target entropy CDF. With this intuition in place, we can further convert ū, into pseudo-entropy value Z that better matches the distribution of the source entropies. This is obtained by applying the inverse source CDF to úg resulting in 2, F- (Q(u,)) Observe that we assume here that F, is invertible, however, in practice, we compute the pseudo-inverse of Fs.
To connect the adaptation scheme presented above to the optimal transport map between the target and source entropies, it is useful to consider an ideal case where we use the log-optimal bet for testing a point null 182. In our case, the null hypothesis is that the distribution the source entropies Z and target entropies Zj is the same, which implies Ho in 3). Following [2), the optimal bet for our null is the true likelihood ratio, formulated as
d.F (zj) dF. (2g)²
(8)
where Fy' is the CDF of the target entropy z To align with 66). we can equivalently write 12 (Z)) as a betting function that gets ú as an input 61, Lemma 1]:
(FF (Q)) = dFF(FF (u₂)) dG,)((u))
62 =2
b2
50pl (ug))
(9)
Notably, this optimal betting function is infeasible to compute in practice, as Fy is unknown. Yet, it implicitly suggests that more powerful betting functions could result in better estimate of the target entropy CDF. Also, the optimal betting function reveals an important property of our adaptation scheme, formally given below.
Let X be fresh sample from the target domain with its corresponding Z} een (Sotu (Xj) and uj= FZ))). Assume F, is invertible and Z is continuous, and suppose the betting function represents the true likelihood ratio O.. Then, the adapted 2 (QuP is the optimal transport map from target to source entropies with respect to the Wasserstein distance.
Proposition 2.
The proof of this proposition builds on 61 Lemma 1] and is provided in Appendix C.4 This result highlights that our online, martingale-bass adaptation scheme grounded in optimal transport principles. This, in turn, provides a principled way to minimize the discrepancy between probability distributions. Leeeaggigg this connection, the entropy matching loss function (2). which we employ to self-train the model, can be understood as minimizing the discrepancy between the entropy
7

---

distributions of the source and target domains. This implies that our loss function aligns the model's predictions across these domains. This connection also explains the "no-harm" effect of the proposed loss. When xxy = yx we get that (u,) (u,) = Wj in the ideal case of (9), implying that Z; = Zj. In practice, considering the betting function from 4 anticipate that cj will be close to zero thanks to our online optimization scheme, which, in turn, results in (u,,) =s uj in
3.5 Putting it all together
AlgorithmP 2 in the Appendix summarizes the entire adaptation process of POEM. This algorithm starts by computing the empirical CDF F, of the source entropies to estimate Fs, using unlabeled holdout samples from the source domain (line 6). The betting and pseudo-entropy estimation steps
are presented in lines 12 14. Observe that line 14 we use the pseudo-inverse of F, obtain Zj. The algorithm then proceeds to adapt the classifier's 's parameters in direction that minimizes our self-super loss (line [5). Specifically, we only update the parameters of the normalization layers of the classifier fotw which is a common practice in the test-time adaptation literature 43]
The self-training step is done by minimizing a variation of the entropy matching loss fmaich
1 < A]
gnalchtt (Zacc (z) = gmalch (z).Zz)
exp {2 - (2) - } where Zj = g00 (fa+w (x²)). (10)
The above loss function includes an additional sample-fittering [zz < A] and sample-weighting 1/ exp(2 (Z) } components, where À > 0 is a pre-defined thresholding parameter. The sample- filtering idea is widely used in this literature 42 43 as the predictions of samples with high entropy examples tend to be inaccurate. Aligning with this intuition, the cample-weighting gives a higher weight to samples with low entropies [42]. Finally, we predict a new betting parameter ej for the next iteration by applying an SF-OGD step (line
4 Experiments
We conduct a comprehensive evaluation of POEM across diverse range of datasets and scenarios commonly used in test-time adaptation literature. Our experiments span ImageNet, ImageNet-C. CIFARIO-C, and SEFAR100-C datasets for evaluating the robustness to shifts induced by corruptions, and the Office-Home dataset for domain adaptation. We study the performance of our method in both single-shift and continual-shift settings. Our evaluation demonstrates that POEM is highly competitive with leading baseline test-time adaptation methods in terms of accuracy and runtime. In the interest of space, this section focuses on the results for the ImageNet dataset, as it is the most challenging one among those considered. Details and results for the experiments on CIFAR and Office-Home datasets are provided in Appendices] F.3 and F.4. respectively.
Throughout this section we use the test set of ImageNet to form our in-distribution dataset, and utilize ImageNet-C which contains 15 different types of corruptions at five increasing severity levels- to simulate various out-of -fstiriiibtttiinnnnn scenarios. Notably, since the images in ImageNet-C are variations of the same images from the ImageNet test set, our experiments simulate a realistic -uittf-distribution test set by including only single corrupted version of each image. To demonstrate the versatility of POEM, we consider two pre-trained classifiers f3 of different architectures: Vision Transformer (ViT) [63] with layer norm (LN) and ResNet50 [64] with group norm (GN). We compare POEM to four leading entropy minimization methods-TENT [41] EATA (42), SAR [43]. and COTTA 197)uusin
code provided by the authors. Importantly, all adaptation methods update only the normalization parameters (LN/GN) of the model, ensuring a fair comparison. Following [K]. we employ a fully online setting with batch size of 1, in which the model is updated after observing a new test sample; see Appendix F22 for implementation details and choice of we defer the results obtained by the ResNet classifier to Appendix F22.1 and focus here on the results obtained by the ViT model.
In the interest of space,
we evaluate our approach in a continual setup in which sudden distribution shifts occur during testing. To simulate this, we create test set of 15,000 samples by randomly selecting 1,000 samples from each corruption type at a fixed severity level (a corruption segment) and concatenating all 15 segments to form the test data. We apply all adaptation methods
Continual shifts Inspired by 197 98
8

---

80
Method
VIT(LN) Acc.
70
No adapt
60
COTTA
FATA
40
POEM (ours)
30
Gruss: Noise
Shat Roise
Impularr Noise
Datocus Bur
Glass Blur
Motion Blur
Zoam Blar
Showe
Frest
Fog
Bright.
Contrst
Blsttic Trans.
Flast.
Joag Camp.
time
Corruption Type (Severity 5)
80
VIT(LN) Acc.
70
56.0
54.0
50
52.0
500
Severity Level (Gaussian Noise)
Severity Level (Gaussian Noise]
Corruption Segment Size
time
time
Figure 2: Continual test-time adaptation on ImageNet-CC with a ViT model. Top: Per-corruption
accuracy with corruption segment size of 1,000 examples. Results obtained over 10 independent trials; error bars are tiny. Bottom left and center: Severity shift-low (1) to high (5) and back to low (left), and high (5) to low (1) and back high (center). To improve the readability here, we only present POEM, the best- best-performing baseline method (EATA), and the no ao-adapt approach. Bottom right: Mean accuracy under continual corruptions function of the corruption segment size.
in combination with a ViT model and present the results in Figure 2 Following the bottom panel in that figure, can that POEM achieves higher accuracy than all of the benchmark methods. Next, we investigate how quickly our method adapts the model to new shifts by varying the segment size of each corruption. As shown in Figure 2 (bottom right), POEM exhibits faster adaptation than the baseline methods. It successfully enhances the accuracy the pre-trained model with as few as 100 examples per corruption (test set of size 1,500) well with longer adaptation using 2,000 examples per corruption (test set of size 30,000). Finally, we explore another realistic scenario of continual adaptation by varying the severity level every 1,000 samples while keeping a fixed corruption type. The bottom left and center panels in Figure [ggure[]show, that POEM outperforms the best baseline method (EATA) in this setting as well. Lastly, similar experiments conducted with ResNet model are presented in Figure ( in the Appendix, showing that our method attains faster adaptation and superior accuracy than the baseline methods.
Single shift We now consider scenario with a single corruption type of fixed severity level, which follows 41 43 99]. Table summarizes the average results across all corruptions for severity level 5, demonstrating that POEM achieves an average accuracy 67.36% for the ViT model, outperforming the best baseline method (EATA) with an absolute average accuracy gap of 3.22%. A detailed breakdown by corruption type for each classifier is provided in Table 2 in the Appendix. Notably, POEM outperforms all benchmark methods on all corruption types for ViT, while achieving higher test accuracy in 9 out of the 15 corruption types for the ResNet model.
In distribution In this setting, we apply all methods on the validation set of the ImageNet dataset. Following in the Appendix, all the methods maintain similar accuracy as the original model, however, the baseline methods tend to increase the expected calibration error (ECE) [68] 100 and unnecessarily modify the model's parameters, as measured by |! In contrast, following Figure (left panel), POEM exhibits minimal changes both for ECE and model parameters, as desired. Figure the Appendix leads similar conclusions for the ResNet model.
Additional experiments on ImageNet, including ablation study The right panel of Figure plots the value of the betting parameter over time, for both in- and out-of-distribution scenarios. Observe how remains near zero under the in-distribution setting, explaining the minimal change in accuracy, ECE, and model's parameters || presented in the left panel of Figure
By contrast, when

---

1.00
Param. change
Method adept TENT
Betting variable
In ditribution
0.75
Severity 1 (bright.)
|0]
0.50
Severity (bright.)
EATA
0.25
SAR
POEM (ours)
0.00
0.010 0.012 0.014
5000
10000
15000
20000
25000
30000
35000
ECE
Example
Figure 3: In-distribution experiment on ImageNet (left panel): calibration error (ECE [100]] versus || a metric that evaluates the classifier's parameters deviation from the original VIT model. Lower values on both axes are better. Results are averaged across 10 independent trials; standard errors and accuracy each method are reported in Table the appendix. The behavior of the betting parameter (right panel): the value of is presented as function of time for both in- and ciat-o--distrbbution experiments (a single shift, two severity levels).
considering out-of out-of-distribun distribution test data single shift, we can see that is high at the beginning and gradually reduces over time, indicating that the self-training process adapts the model the new environment. A similar behavior is observed for the ResNet model; see Figure in the Appendix. This conclusion is further supported by Figure in the Appendix, showing that the CDF of the target entropies of the adapted ResNet model nearly matches the CDF of the source entropies. In Figure the Appendix we show how the martingale can powerfully detect shifts-ever a minor one (brightness with severity level I)-and also show how our adaptation mechanism gradually limits the martingale's 's growth. Lastly, we conduct an ablation study, comparing the test accuracy of POEM using two loss functions: emarch ( and gmatcho+ (10). Table in the Appendix presents these results for single shift scenario, averaged over 15 corruptions of ImageNet-C at the highest severity level. Both losses improve the original model accuracy, with gmatch++ showing better adaptation performance.
Experiments CIFAR-10C, CIFAR100-C, and OfficeHome datasets, sensitivity study, and runtime comparisons Appendices F33 and F4 present experiments on these additional datasets, leading similar conclusions about the competitive adaptation accuracy of POEM compared to baseline methods. Notably, Appendix F33 includes experiments on both relatively short and long adaptation streams, with lengths of 15,000 and 112,500 samples, respectively. These experiments also include a sensitivity study on the learning rate 7) used for self-training the model, showing that POEM exhibits greater stability across different learning rate values compared to SAR and TENT. Additionally, these experiments show that POEM'S runtime is comparable to TENT and EATA, and faster than SAR.
5 Discussion
We introduced a novel, martingale- based approach for test-time adaptation that drives the test-time entropies of the self-trainee model to match the distribution of source entropies. We validated our approach with numerical experiments, demonstrating that: (i) under in-distribution settings, POEM maintains the performance of the source model while avoiding overconfident predictions, a crucial advantage over entropy minimization methods; (ii) in relatively short out-of distribution test periods, approach achieves faster adaptation than entropy minimization methods, which is attributed to our betting scheme that quickly reacts to distribution shifts; and (iii) in extended test periods, POEM achieves comparable adaptation performance and stability to strong baseline methods.
One limitation of our method is the requirement for holdout unlabeled source domain data to estimate the source CDF. Notably, this CDF is pre-computed offline, and at test time we do not require additional access to source data, similar to EATA's 's requirements. Another limitation our approach is the choice of hyperparameters, particularly the self-training learning rate 77. However, our sensitivity study showed that our method is fairly robust to this choice, especially compared to baseline methods. Lastly, similar to other experimental works, we anticipate that POEM may fail to improve accuracy in settings that we have not explored, especially when facing an aggressive shift. Yet, our monitoring tool can detect such distribution shifts, which is an important mechanism that does not appear in other test-time adaptation methods.
Future directions are discussed in Section G of the Appendix. Lastly, we note that while the goal of this paper is to enhance the robustness of MI. to unseen environments, there are many potential socictal consequences of our method, similar to other works that aim to advance this field.
10

---

6 Acknowledgments and Disclosure of Funding
This research was supported by the ISRAEL SCIENCE FOUNDATION (grant No. 729/21). Y. R. thanks the Career Advancement Fellowship, Technion.
References
[1] Qin Wang. Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7201-7211, June 2022.
[2] Thomas Varsavsky. Mauricio Orbes-Arteaga, Carole H Sudre, Mark S Graham, Parashkev Nachev, and M Jorge Cardoso. Test-time unsupervised domain adaptation. In International Conference in Medical Image Computing and Computer Assisted Intervention, pages 228-436. Springer, 2020.
[3] Abolfazi Farahani, Sahar Voghoei, Khaled Rasheed, and Hamid Arabnia. brief review of domain adaptation. Advances in data science and information engineering, pages 877-894, 2021.
[4] Garrett Wilson and Diane Cook.
survey of unsupervised deep domain adaptation. ACM Transactions Intelligent Systems and Technology (TIST), 11(5)pp. 1 1-46, 2020.
[5] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the International Conference on Machine Learning, pages 1180-1189. PMLR, 2015.
[6] Vishal M Patel, Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. Visual domain adaptation: A survey of recent advances. IEEE signal processing magazine, 32(3):pp. 53-69, 2015.
Wouter M Kouw and Marco Loog. An introduction to domain adaptation and transfer learning. arXiv preprint arXiv: 11806 2018.
[8] Xingchao Peng, Ben Usman, Neela Kaushik, Dequan Wang, Judy Hoffman, and Kate Saenko. Visda: A synthetic-to-real benchmark for visual domain adaptation. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition Workshops, pages 2021-2026. 2018.
[9] Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In Proceedings of the International Conference on Machine Learning, pages 819-827. PMLR, 2013.
[10] Vidit Jain and Erik Learned-Mller. Online domain adaptation of pre-trained cascade of classifiers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 577-584. IEEE, 2011.
[11] Xiaofeng Liu, Chaehwa Yoo, Fangxu Xing. Hyejin Oh, Georges El Fakhri, Je-Won Kang, and Jonghye Woo. Deep unsupervised domain adaptation: A review of recent advances and perspectives. APSIPA Transactions Signal and Information Processing, 11, 2022.
[12] John Blitzer, Ryan McDonald, and Fernando Pereira. Domain adaptation with structural correspondence learning. Proceedings Conference on Empirical Methods in Natural Language Processing, pages 120-128, 2006.
[13] Yongjie Shi, Xianghua Ying, and Jinfa Yang. Deep unsupervised domain adaptation with time series sensor data: A survey. Sensors, 22(15), 2022.
[14] Joaquin Cidooero-Canndllee Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence. Dataset Shift in Machine Learning. The MIT Press, 2009.
[15] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference Learning Representations, 2018.

---

[16] Jeya Maria Jose Valanarasu, Pengfei Guo, Vibashan VS, and Vishal M. Patel. On-the-fly test-time adaptation for medical image segmentation. arXiv preprint arXiv:2203.05574, 2022.
[17] Yanyu Ye, Zhenxi Zhang, Wei Wei, and Chunna Tian. Multi task consistency guided source-free test-time domain adaptation medical image segmentation. arXiv preprint arX1v:2310.11766. 2023.
[18] Neerav Karani, Ertunc Erdil, Krishna Chaitanya, and Ender Konukoglu. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis, 68: 68:101907 2021.
[19] Yufan He, Aaron Carass, Lianrui Zuo, Blake E Dewey, and Jerry L Prince. Autoencoder based self-supervised test-time adaptation for medical image analysis. Medical Image Analysis, 72:102136, 2021.
[20] Zhang Li, Zheng Zhong, Yang Li, Tianyu Zhang, Liangxin Gao, Dakai Jin, Yue Sun, Xianghua Ye, Li Yu, Zheyu Hu, Jing Xiao, Lingyun Huang, and Yuling Tang. From community-aqquire pneumonia to COVID-19: a deep learning-based method for quantitative analysis of COVID- 19 thick-section CT scans. European Radiology, 50(12):6828-668 2020.
[21] Jian Liang, Ran He, and Tieniu Tan. comprehensive survey on test-time adaptation under distribution shifts. arXiv preprint arXiv:2303. 153661 2023.
[22] Prashant Pandey, Mrigank Raman, Sumanth Varambally, and Prathosh AP. Generalization on unseen domains via inference-time label-preserving target projections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12924-12933, June 2021.
[23] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of- distrribibtton generalization. ICCV, 2021.
[24] Marvin Zhang, Sergey Levine, and Chelsea Finn. MEMO: Test time robustness via adaptation and augmentation. Advances in neural information processing systems, 35:38629-3866 2022.
[25] Xuefeng Hu, Gokhan Uzunbas, Sirius Chen, Rui Wang, Ashish Shah, Ram Nevatia, and Ser-Nam Lim. Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint 10.1 2021.
[26] Saypraseuth Mounsaveng, Florent Chiaroni, Malik Boudiaf, Marco Pedersoli, and Ismail Ben Ayed. Bag of tricks for fully test-time adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1936-19455 2024.
[27] Eric Mintun, Alexander Kirillov, and Saining Xie. On interaction between augmentations and corruptions in natural corruption robustness. Advances in Neural Information Processing Systems, 34:3571-3588 2021.
[28] Liang Chen, Yong Zhang, Yibing Song. Ying Shan, and Lingqiao Liu. Improved test-time adap- tation for domain generalization. In Proceedings of the IEEE//VV Conference on Computer Vision and Pattern Recognition, pages 1112224182,, 2023.
[29] Minmin Chen, Kilian Q Weinberger, and John Blitzer. Co-training for domain adaptation. Advances in neural information processing systems, 24:2456-2466 2264,,24,,7 2011.
[30] Yuanyuan Xu, Meina Kan, Shiguang Shan, and Xilin Chen. Mutual learning of joint and separate domain alignments for multi-source domain adaptation. In WACV, pages 1890-1899, 2022.
[31] Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference computer vision (ECCV), pages 289-305, 2018.
12

---

[32] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In Proceedings of the International Conference on Machine Learning, pages 9248..248. PMLR, 2020.
[33] Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
[34] Yuejiang Liu, Parth Kothari, Bastien van Delfi, Baptiste Bellot-Guriet, Taylor Mordan, and Alexandre Alahi, TTT++: When does self-supervised test-time training fail or thrive? Ad- vances in neural information processing systems, 2021.
[35] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in neural information processing systems, 17, 2004.
[36] Philipp Haeusser, Thomas Frerix, Alexander Mordvintsev, and Daniel Cremers. Associative domain adaptation. In Proceedings of the IEEE international conference on computer vision, pages 2765-2773 2017.
[37] Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko. Simultaneous deep transfer across domains and tasks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 4078-4076, 2015.
[38] Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo. AutoDIAL: Automatic domain alignment layers. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 5077-5085, 2017.
[39] Kuniaki Saito, Yoshitaka Ushiku, and Tatsuya Harada. Asymmetric tri-training for unsu- pervised domain adaptation. In Proceedings of the International Conference on Machine Learning, pages 2988 299822997.200 2017.
[40] Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. DIRT-T approach to unsu- pervised domain adaptation. In Proceedings of the International Conference on Learning Representations, 2018
[41] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. International Conference on Learning Representations, 2020.
[42] Shuaicheng Niu. Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In Proceedings of the International Conference on Machine Leurning, pages 16888-16905, 2022.
[43] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations, 2023.
[44] Masud An-Nur Islam Fahim and Jani Boutellier. SS-TTA: Test-time adaption for self- supervised denoising methods. In IEEE/CVF Conference Computer Vision and Puttern Recognition Workshops (CVPRW), pages 1178-1187 2023.
[45] Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adapta- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 295-305, 2022.
[46] Tomer Cohen, Noy Shulman, Hai Morgenstern, Roey Mechrez, and Erez Farhan. Self- supervised dynamic networks for covariate shift robustness. arXiv preprint arXiv:2006.03952. 2020.
[47] Alexander Bartler, Andre Bühher,, Felix Wiewel, Mario Döbler, and Bin Yang. Mt3: Meta test-time training for self-supervised test-time adaption. International Conference on Artificial Intelligence and Statistics, pages 3080-3090, 2022.
13

---

[48] Alexander Bartler, Florian Bender, Felix Wiewel, and Bin Yang. TTAPS: Test-time adaption by aligning prototypes using self-supervision. In IEEE International Joint Conference on Neural Networks (IJCNN, pages 1-8, 2022.
[49] Hao Zhao, Yuejiang Liu, Alexandre Alahi, and Tao Lin. On pitfalls of test-time adaptation. In International Conference on Machine Learning (ICML), 2023.
[50] Yi Su, Yixin Ji, Juntao Li, Hai Ye, and Min Zhang. Beware of model collapse! fast and stable test-time adaptation for robust question answering. In Conference on Empirical Methods in Natural Language Processing. 2023.
[51] Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottanamurak, and Sung-Ju Lee. SoTTA: Robust test-time adaptation on noisy data streams, Advances in Neural Information Processing Systems, 36, 2024.
[52] Ori Press, Ravid Shwartz-Ziv, Yann LeCun, and Matthias Bethge. The entropy enigma: Success and failure of entropy minimization. arXiv preprint urXiv:2405.05012, 2024.
[53] Xiaofu Wu, Suofei Zhang, Quan Zhou, Zhen Yang, Chumming Zhao, and Longin Jan Latecki. Entropy minimization versus diversity maximization for domain adaptation. IEEE Transactions Neural Networks and Learning Systems, (4(6))2896-990)) 2021.
[54] Pietro Morerio, Jacopo Cavazza, and Vittorio Murino. Minimal-entropy correlation align- ment for unsupervised deep domain adaptation. In International Conference on Learning Representations, 2018
[55] Glenn Shafer, Alexander Shen, Nikolai Vereshchagin, and Vladimir Vovk. Test martingales, bayes factors and p-values. Statistical Science, 26(1). February 2011.
[56] Vladimir Vovk, Ilia Nouretdinov, and Alexander Gammerman. Testing exchangeability on-line. Proceedings of the 20th International Conference on Machine Learning (ICML), pages 768-775, 2003.
[57] Vladimir Vovk, Ivan Petej, and Alex Gammerman. Protected probabilistic classification. In Conformal and Probabilistic Prediction and Applications, pages 297-299, 2021.
[58] Francesco Orabona and Dávid Pál. Scale-free algorithms for online linear optimization. In International Conference on Algorithmic Learning Theory, Phees 287-301, 2015.
[59] Francesco Orabona and Dávid Pál. Scale-free online learning. Theoretical Computer Science, 716:50-61 69, 2018.
[60] Francesco Orabona and Dávid Pál. Coin betting and parameter- free online learning. Advances in Neural Information Processing Systems, 29:577-585, 2016.
[61] Vladimir Vovk. Protected probabilistic regression. Technical report, Tech. Rep, 2021.
[62] Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
[63] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
[64] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-778. 2016.
[65] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2018.
[66] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference computer vision and pattern recognition, pages
2017.
14

---

[67] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation. IEEE transactions on pattern analysis and machine intelligence. 39(9):1853-1865 2016.
[68] Eungyeup Kim, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Reliable test-time adaptation via agreement-on-thh--ine. In NeurIPS Workshop on Distribution Shifts: New Frontiers with Foundation Models, 2023.
[69] Glenn Shafer and Vladimir Vovk. Game-theoretic foundations for probability and finance. Wiley Series in Probability and Statistics, 2019.
[70] Shubhanshu Shekhar and Aaditya Ramdas. Nonparametrie two-sample testing by betting. IEEE Transactions on Information Theory, 2023.
[71] Aleksandr Podkopaev and Aaditya Ramdas. Sequential predictive two-sample and indepen- dence testing. Advances in Neural Information Processing Systems, 36, 2024.
[72] Shalev Shaer, Gal Maman, and Yaniv Romano. Model-X sequential testing for conditional independence via testing by betting. In International Conference on Artificial Intelligence and Statistics, pages 2054-2086, 2023.
[73] Peter Grünwald, Alexander Henzi, and Tyron Lardy. Anytime-valid tests of conditional independence under model-X. Journal of the American Statistical Association, pages 1-12, 2023.
[74] Valentina Fedorova, Alex Gammerman, Ilia Nouretdinov, and Vladimir Vovk. Plug-in martin- gales for testing exchangeability on-line. In Proceedings of the International Conference on Machine Learning, pages 923-930 2012.
[75] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Testing exchangeability. In Algorithmic Learning in a Random World, pages 227-263. Springer, 2022.
[76] Boyan Duan, Aaditya Ramdas, and Larry Wasserman. Interactive rank testing by betting. In Conference on Causal Learning and Reasoning, pages 201-235, 2022.
[77] Aaditya Ramdas, Johannes Ruf, Martin Larsson, and Wouter M Koolen. Testing exchangeabil- ity: Fork-convexity, supermartingales and e-processes. International Journal of Approximate Reasoning, 141:83-109, 2022.
[78] Aytijhya Saha and Aaditya Ramdas. Testing exchangeability by pairwise betting. In Interna- tional Conference on Artificial Intelligence and Statistics, pages 4915-4923, 2024.
[79] Peter Grünwald, Rianne de Heide, and Wouter M Koolen. Safe testing. In IEEE Information Theory and Applications Workshop (ITA), pages 1-54, 2020.
[80] Glenn Shafer, Alexander Shen, Nikolai Vereshchagin, and Vladimir Vovk. Test martingales, bayes factors and p-values. Statistical Science, 26(1):84, 2011.
[81] Vladimir Vovk and Ruodu Wang. E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736-1754, 2021.
[82] Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, and Glenn Shafer. Game-theoretic statisties and safe anytime-valid inference. Statistical Science, 38(4)5576-601, 2023.
[83] Steven R Howard, Aaditya Ramdas, Jon McAuliffe, and Jasject Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2). 2021.
[84] Ian Waudby-Smine and Aaditya Ramdas. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2023.
[85] Peter D Grünwald. The e-posterior. Philosophical Transactions of the Royal Society A,
381 381(2247). 2023.
[86] Muriel Felipe Pérez-Ortiz, Tyron Lardy, Rianne de Heide, and Peter Grünwald. E-statisties, group invariance and anytime valid testing, arXiv preprint ArXivv2208.076111 2022.
15

---

[87] Wouter M Koolen and Peter Grünwald. oog-optimal mytimee-vliie e-values. International Journal of Approximate Reasoning, 141:69-82, 2022.
[88] Vladimir Vovk and Ruodu Wang. Confidence and discoveries with e-values. Statistical Science, (8(2):329-335) 2023.
[89] Shubhanshu Shekhar and Aaditya Ramdas. Reducing sequential change detection to sequential estimation. arXiv preprint arXiv:2309.09111, 2023.
[90] Jaehyeok Shin, Aaditya Ramdas, and Alessandro Rinaldo. E-detectors: A nonparametric framework for sequential change detection. The New England Journal of Statistics in Data Science, pages 1-32, 2023.
[91] Vladimir Vovk. Testing randomness online. Statistical Science, 36(4):595 611, 2021.
[92] Charalambos Eliades and Harris Papadopoulos. A conformal martingales ensemble approach for addressing concept drift. In Conformal and Probabilistic Prediction with Applications, volume 204, pages 328-33882 346.. PMLR, 2023.
[93] Vladimir Vovk. Testing for concept shift online. arXiv preprint arXiv:2012.14244 2020.
[94] Liang Dai and Mohamed-Raie Bouguelia. Testing exchangeability with martingale for change-poing detection. International Journal of Ambient Computing and Intelligence (LJACI), 12(2):1-20, 2021.
[95] Vladimir Vovk, Ivan Petej. Ilia Nouretdinov, Ernst Ahlberg, Lars Carlsson, and Alex Gam- merman. Retrain or not retrain: Conformal test martingales for change-point detection. In Conformal and Probabilistic Prediction and Applications, pages 191-210. PMLR, 2021.
[96] Paul Lévy. Théorie de I' I'addition de variables aléatoires. second edition 1954. (gauthier-villars. paris). The Mathematical Gazette, 39, 1955.
[97] Qin Wang. Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7201-7211, 2022.
[98] Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages I 15922-15932, 2023.
[99] Zixin Wang, Yadan Luo, Liang Zheng, Zhuoxiao Chen, Sen Wang, and Zi Huang. In search of lost online test-time adaptation: A survey. arXiv preprint arXiv:2310.20199 2023.
[100] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321-1330. PMLR, 2017.
[101] Menglong Lu, Zhen Huang, Zhiliang Tian, Yunxiang Zhao, Xuanyu Fei, and Dongsheng Li. iitta-ssalis--entroy minimization: a new self self-training approach for domain adaptation on text classification. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 5159-5169, 2023.
[102] Hong Liu, Jianmin Wang, and Mingsheng Long. Cycle self-training for domain adaptation. Advances in Neural Information Processing Systems, 34:22968-22988 2021.
[103] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges representation learning, ICML, volume 3, page 896, 2013.
[104] Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J 7ico Kolter. Test time adaptation via conjugate pseudo-labels. Advances in Neural Information Processing Systems, 13:6204-6218 2022.
[105] Jun-Kun Wang and Andre Wibisono. Towards understanding GD with hard and conjugate pseudo-labels for test-time adaptation. In International Conference on Learning Representa- tions, 2022.
16

---

[106] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adapta- tion. Advances in neural information processing systems,
2020.
[107] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International PMLR, 2020.
conference machine learning, pages
[108] Xinyang Chen, Sinan Wang, Jianmin Wang, and Mingsheng Long. Representation subspace distance for domain adaptation regression. In ICML, pages 1749-1759, 2021.
[109] Karsten M Borgwardt, Arthur Gretton, Malte Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22( (1)))44999000 -57, 2006.
[110] Aadyot Bhatnagar, Huan Wang, Caiming Xiong, and Yu Bai. Improved online conformal prediction via strongly adaptive online learning. In Proceedings the International Conference Machine Learning, pages 2337-2363. PMLR, 2023.
[111] Victor M Panarctos and Yoav Zemel. Statistical aspects of Wasserstein distances. Annual review of statistics and its application, 6:405-431, 2019.
[112] Jean Ville. lere these: Etude critique de la notion de collectif: 2eme these: La transformation de Laplace. PhD thesis, Southier-Villant & Cie, 1939.
[113] Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. Note: Robust continual test-time adaptation against temporal correlation. Advances in Neural Information Processing Systems, 35:27253 27266. 2022.
[114] Aleksandr Podkopaev and Aaditya Ramdas. Distribution- free uncertainty quantification for classification under label shift. In Uncertainty in artificial intelligence, pages 844-853. PMLR, 2021.
[115] Zhi Zhou, Lan-Zhe Guo, Lin-Han Jia, Dingchu Zhang, and Yu-Feng Li. Ods: Test-time adaptation in the presence of open-world data shift. In International Conference Machine Learning, pages 42574-42588 PMLR, 2023.
17

---

A Additional related work: test-time adaptation
As discussed in the main manuscript, entropy minimization is known to be effective for test time adaptation. However, the works in [4 4 demonstrate that samples with high entropy loss can lead to noisy or overly aggressive updates of model parameters. To address this issue, 44 43 filter out high-entropy samples, and [43] also employs gradient clipping to stabilize self-training. These algorithmic modifications emphasize the importance of controlling the minimization of the entropy loss, which greatly aligns with the goal of our work. In our framework, we also use entropy as guiding force for self trainng. However, instead of directly minimizing the entropy, our approach focuses on matching the distribution of the entropy losses at test time with that of the source domain. Notably, our method is versatile and can accommodate alternative choices beyond entropy, such as Tsallis entropy [10] 102 or cross-entropy evaluated with a pseudo-label [103] [05] in place of the unknown true label. We leave the exploration of these alternative options for future work.
Another concern in test-time adaptation frameworks is the continuous learning mechanism, which often leads performance degradation on in-distribution data. To address this challenge, EATA 42] introduces an anti-forgetting strategy that optimizes the model by focusing on the reliability of samples and incorporates a weight regularizer further improve stability. In our work, instead of relying on weight regularization, we preserve in-distribution performance through "no-harm" approach, ensuring minimal model updates where no distribution shifts have occurred.
crucial aspect of self-training is the selection of model parameters w to update. A widely adopted practice is to update the parameters of the normalization layers. This constraint plays a vital role in mitigating overfitting, which is imperative to prevent the model from collapsing and making trivial predictions. The TENT 441 approach demonstrates that minimizing entropy by modifying only the batch normalization parameters can significantly enhance out-of out-of-distribution performance. However, TENT requires working with large batches, which can limit the ability handle mixed-distribution shifts within a single batch-for instance, a batch containing a subset of blurred images, another subset of noisy images, and so on. To alleviate this issue, the SAR method suggests updating the group or layer normalization parameters, unlocking the use of smaller batch sizes. In turn, this adjustment enhances the model's s adaptivity under distriibiitiittonnnnn shifts. In our work, we follow this line of research and update the layer normalization parameters: however, we optimize a completely different loss function, aiming to match the distributions of the source and target entropies. While there are works that suggest matching between the source and target distributions [10) this alignment is often affected by the batch size. To stress this point, one might consider using an out-of-the-box distributional matching loss function (e.g.. Maximum Mean Discrepancy [109] to align the source and target entropy distributions, however, this approach requires large batches to obtain an effective estimation. Our approach accumulates the evidence for distribution shift in an online fashion, allowing us to use small batches, even of size one as we do in our experiments.
The work in (97) tackles the challenge of test-time adaptation where a pre-trained model must adjust to continuously changing target domain during inference, without access to original training data. This approach, named COTTA, utilizes veight-averagee and sugmentation-avergn improve label accuracy and reduce error accumulation. To alleviate catastrophic forgetting, CoTTA intermittently reverts some neurons to their initial states. In contrast with CoTTA, our method operates under the assumption of no access to snarformatioossaagmnntttiiionn during testing. Employing such augmentations may further improve the performance of POEM, and we leave this exploration for future work.
pseudo-labels to
B Learning the betting variable online
In this section we present and analyze our online approach to learn the betting variable e,, which relies on the SF-OGD algorithm [59]. The complete algorithm is presented in Algorithm I
In the following analysis, we assume that the betting variable that attains the fastest growth of the capital in hindsight, after observing w,, is in the range [- D, D) c [-2, 2.. We denote this variable by E, E (--D, + D), which can be computed via a simple closed-form expression, given by
E,- = D sign( uu - 0.5).
(11)
With this in place, we define the awip-aware loss function, which we will use to formulate the update rule for e, that maximizes the capital:
18

---

[og + E, (ur 0.5)) if E, 0 and |,,1 D log(1 [ + te, (u, - 0.5) otherwise.
6(),,,)) =-
(12)
In plain words, when ér is out of range but has the same sign as E,, we clip the betting variable with the maximal value allowed (D or DD that would increase the wealth. Otherwise, the loss equal to the log of the bet, obtained with Er.
At each step of the algorithm, first predict the value of e,, then observe ur, which allows to compute E,. Therefore, after observing u, we can compute the (sub)gradient of [2)
V. (.I,,,,)7)) - (u, 0.5)/(1 cu, 0.5)) otherwise.
if Er er > 0 and |er|> D
(13)
Armed with V.L(Er, 67) we are ready to perform the SF-OGD step, formulated as: V.L(Er.G.) c++ = 7 E (VeLe(E,,,))
(14)
where y is learning rate. Lemma (eblow states that e, is indeed bounded.
Lemma 1. The SF-OGD algorithm with a learning rate 0 < < 2 D and initialization for all S j.
[-D 7, D 7 satisfies e, E E--D-
Proof is in Appendix C.2 Following Lemma 1 conclude that must set the learning rate y in the range < < valid betting function
D to ensure that e, is in the range (-2, 2). The latter crucial to formulate a
Building on the analysis of SF-OGD, the following proposition states that this algorithm achieves regret bound on the loss in 12 where the regret function is defined as
Reg(j) Σ ((E,,))))
sup Σ E(E,Ee). eEII-D,D r=1
(15)
Above, ex is the betting parameter that minimizes the loss in hindsight, over the time horizon 1 istsj.
Theorem 1. [Theorem 4 by Orabona and pål [59]; Proposition A.2 by Bhatnagar et al. (1OI
Suppose that c* E I- D). Then, SF-OGI with E, (-D.DD for all I S j. learning rate 0 <<> - D, and any initialization e E I y, D + 7 achieves
1
Vt VI )- A565A
Reg(() S (7 (2D 7) +)) 27
The above result states that for any interval size 1 < 1 S j. the regret of SF-OGD defined (15). bounded by the square-rood of the interval size Vt, divided by the difference between the boundaries of the entire domain [-2, 2] and of the actual 6, domain [D 7. D + 7].
Proof is Appendix C.3
C Proofs
C.1 Proof of Proposition I
Proof. Sy 2 0 for all j E N since u, E [0, 1] and E [-2 by definition.
(S, j N) is martingale under Ho since
Eng [S, S1, Sy-1l Sy-1 En, (b(u,) S1, ..., Sq-1]
= =Sy-1 (1 + Ej EH, [u; - 0.5 S1,..., Sy-1)) = S
-S., and the latter since where the second transition holds since e, depends only on (Si, Sy-1), Exo [ug 0.5| Si,..., S1,...,Sy-11 Sy-1] 0.
D
19

---

C.2 Proof of Lemmal 1
Proof. Leveraging the symmetry of the problem, it suffices to study the setting in which 6 0. Also, without loss of generality, we assume that V (Ei. c1) & 0 for the first iteration of the algorithm; if this does not hold we can always remove these samples until reaching an observation with non-zero gradient. To prove the result, we consider the following cases that can occur when optimizing (12)
Case 1: e, D.
Case 6 > and 6 Frr 0.
Case 3: e, > and e, E, S 0.
We start by analyzing Case 1. Recall that 0 S u, I 1 and that E, (-,, Now, following we can conclude that the gradient of the loss L(E,, e,) ) is bounded V. V.L(E,,E,) E 2- 2-17 By recalling the update rule in (14). we get:
13
V.((E,,eT)
1 - G =7
1=1 (E,)) S7.
(16)
With this in place, can conclude that if Er+1 Er, then
c++1 S e + D+7-
Otherwise, +,+1 E,, then
D 2 Er 2 6T4 Er 7 -7
Above, we used the fact that 0 S 6, < D under Case 1. In sum, we showed that S 6T41 S D+7
We now turn to analyze Case 2. Under the assumptions of this case V.L(Er, = 0 and thus c++1 = c,, ie., the value of the betting parameter will not be modified. To bound the value of 6y+1+ note that this scenario, we already know that Y I E, S D + y. We can also reach Case 2 at the initialization, but then E <= D 7 and thus bounded. Lastly, another entry point to Case 2 is from Case 3, however, we show below that the latter satisfies that
encounter e, D as result of updating the betting parameter in Case 1, but in
S 6 I D D
Lastly, we study Case 3, which can be reached from Case 1 or Case 2. However, in Case 2 c,+1 Cr. so we can concentrate only on the scenario where Case 3 is reached from Case 1, but we already showed that < D + y in this case. Lastly, we can face Case 3 when éJ > D. however, it is bounded 6 <= D + y by the Lemma's ' assumption. Hence, following (13). the gradient bounded VeLo(EE, 4 [- 2 D b-b- Further, the loss can be improved only by reducing the value of er, and thus the SF-OGD step would result in Er+1 S E,. This implies that &, 6741 7, according
to (16). In turn, by recalling that + 2 C > D we conclude that
D+1 7 2 e, Z e++1 2 c - 7 D - 7-
To summarize, the analysis of the cases above indicates that c, E E[-D- D D+ for all 1 <
as desired.
C.3 Proof of Theorem
Proof. Recall that want prove that
Vi Σ (V.L(T,,G)) < 2 (2-b-r), vistsj.
(2D + 7))) )) Reg(t) I (7 + 2
The second inequality holds by following the proof of Lemma showing that V V.L, c)) 1
The proof of the first inequality is a direct consequence of [59 Theorem 4] or
D- 2-D--y1 110 Proposition A.2], and thus omitted. Notable, we can directly invoke [59 Theorem 4] as (i) the
2- 2-
loss function (-) [(FF;)) in (2) 2 is convex; and (ii) the betting variable e, E [-D D - 7,
is bounded for all 1 < S due Lemma] I
20

---

C.4 Proof of Proposition 2
Proof. Since we assume that F5 is invertible and 2 is continuous, we can conclude that Fs is smooth bijection function. This allows us invoke [61 Lemma 1], which states that if F% is the CDF corresponding to the density function d'F', and the mapping F, is smooth bijection, then the CDF Q'P F. (PF) in With this in place, we can write
2 = 11=F-'QQ00 (u,)) = F (F. (F, ==)
where the second equality holds due 66 Lemma 1] and the third equality stems from the definition of uj. being 4 F. (Z)). We conclude the proof by observing that the mapping 2; = (F.' (Z)) is the optimal transport map from the target to the source distribution w.r.t. the Wasserstein distance.
This because 2 is continuous, one-dimensional random variable, with an invertible CDF
F,
D
Using martingales to detect distribution shifts
Martingale (capital)
10
Out of distribution, adaptation
Out-of distribution, online adaptation
10
didditibbution
1034
10a
1.5
1.0
0.5
0.0
5000
10000
15000
20000
25000
30000
35000
Time Step
Figure 4: Martingale behaviour with and without adaptation and on in-distribution data. Visualization of three scenarios: (1) out-of-distribution data (ImageNet-C, brightness level 1) without adaptation, (2) the same out-of--istribution data with online adaptation, and (3) in-distribution data (ImageNet) all ResNet50. The top panel shows the martingale value, that is, the accumulated capital (in log scale) over time, while the bottom panel shows the corresponding betting variable e.
Recall that we established in Proposition validity of the martingale defined in (4). In this section, we describe how a valid test martingale can be used to detect distribution shifts with a type-I error guarantee. This this end, consider test martingale sequence denoted by {S, : No} under the null hypothesis Ho of no distribution shift (3). Ville's inequality [112] plays crucial role in bounding the probability of this martingale exceeding a specific threshold under Ho. Specifically, for any value of a between 0 and Ville's 's inequality states the following:
IHo 3j 1: 1; 2 S alEE [So] =
(17)
Suppose we set significance level a = 0.01. The above incquality states that under the assumption of no distribution shift, the martingale's 's value will exceed threshold of 1/a (in this case, 100) with a probability of at most a = 0.01. This bound on the probability allows us to simultaneously control the type-I error across all time steps. This implies that can declare that we face a distribution shift the martingale value pass the threshold 1 1/a. For instance, we set the threshold to 100 or higher, the type-I error is guaranteed to be less than equal to 0.01.
FigureFftop panel offers visual representation of the martingale's behavior under different scenarios; see Appendix FF for implementation details. As discussed above, under the null hypothesis of no distribution shift, the martingale is expected remain lower than with high probability. This precisely what we observe in the in-distribution data acenario-the martingale value remains under
21

---

1. Here, we consider the ImageNet dataset, where we applied pre-trained ResNet50 model on the ImageNet validation set.
In contrast, when the pre-trained model is applied to corrupted, out-of-distribution data (ImageNet-C) the martingale value deviates significantly from 1, reaching levels of up 10200, This substantial increase validates the presence of a distribution shift. Interestingly, Figure 4 also reveals the effec- tiveness of our online adaptation. The adaptation process gradually limits the martingale value from growing compared to the non-adaptation case. Observe how the martingale of the adapted model eventually converges to plateau. This visually demonstrates the success of adapting the model to the new distribution, making the target entropies statistically indistinguishable from the source data.
E Algorithms
In this section, we present a series of algorithms essential to understanding and implementing our proposed methods. is important note that throughout these algorithms, do not explicitly state each instance where lists or variables are updated; however, such updates are implicilly understood to occur during computations.
We use the "Assume" directive in our algorithms to outline which variables are accessible as global variables. These global variables are updated as part of the algorithms' operations but not repeatedly declared within each algorithmic step. This approach is chosen to streamline the presentation and focus the algorithmic logic rather than the mechanics of data handling.
Algorithm SF-OGD Step
Require:
My [0, 1]
Assume:
ej: last betting parameter's value
co) V.L(E)-1, {)-)) past gradient values of the log loss from
12
D: betting variable clip value
7y SF-OGD learning rate
1: E + D sign(u; 0.5)
Following
11
2:
if E, and 1e > D then
3:
C,,(E,,C))) +0
Following
[3
4: else
5:
V.L(F.j.) +-
u U.5
> Following
13
6: end if
(.E
7: 6j+1 + - (V..L. ((E.,)))))))
Following
14
8: return Cj+1
22

---

Algorithm 2 Protected Online Entropy Matching (POEM)
Require:
D {Xj)ド=ı holdout data from source distribution
fg. pretrained model
D: betting parameter's value
?y: SF-OGD learning rate
q) model learning rate
X: entropy filter threshold, see
10
1: Init: é1 = =0, <- 0
We update only the network's norm. layers
2: Compute empirical CDF function source entropies
3: for X in D do
4: Z < g00 (fa(())
5: end for
6: Define: F.(:) = 2 Ex 112 & =}
Empirical CDF function
7: Online adaptation
8: j 1
9: while Get test sample X do
Zj + gen (fotw, (xx))
Compute test entropy
4j (Z))
> Apply probability integral transform
b, 1 + (uj 0.5)
b Place bet against the null (3)
J 5 + Uj (1 2/
> Adapt u, according to 7
14:
Z, - {min Pz(2)= 2 úg in
Transport to source domain, (ū))
15:
Wj+1 +- Wj emancht+ (Z), Mn)
Update norm. layers' params. according to (10)
16:
g-1 Alg.
Update the betting variable
17:
j+j+1
end while
F
Supplementary experiments
F1 Synthetic experiment
To evaluate the methods, we generate synthetic dataset by generating sets of points, each containing 2, 500 samples. Specifically, samples for the first set (class -1) are drawn from a normal distribution NN-1, 1), while samples for the second set (class +1) are drawn from W1,, 1). We use these 5, 000 samples to calculate the source CDF. Note that we do not use a training set, as we initialized the pre-trained model's parameter the optimal value, which is = 0.
We then create two test sets:
In-distribution test data, which consists of 10,000 samples per class, following the source distribution.
Out-of-distribation test data, which consists of 10,000 samples per class, but with shifted distributions the class -1 samples are drawn from (0, 1) and the class samples are drawn from (2, 1). This represents a distributional shift of unit in X.
We defined our model parameter as w E IR. Two types of loss functions were employed:
Entropy loss gent computed by evaluating the entropy of each point's prediction given the model parameter w.
Matching loss gmatch (2) computed using an optimal transport mapping, i.e., F-1 F2, to match the set of target entropies derived from fa to the source entropies obtained from the holdout data.
Hyperparameters and training scheme The optimization each method was executed over 200 steps using fixed learning rate of 5. Although lower learning rates were also effective, higher
23

---

learning rate was chosen to accelerate the demonstration. The optimization was performed with batch size of 64. Each method's performance was evaluated the end of this process. We clipped entropy minimization convergence if it went outside the bounds of our plot. In all experiments conducted in this paper, we choose the following set of hyperparameters, defined Algorithm
= 1.8.
7 v.v 0.0722.
F.2 ImageNet-C experiments
Models Our experiments utilize two pre-trained architectures: Vision Transformer (ViT) with layer normalization (LN) and ResNet50 with group normalization (GN). Both are pre-trained models from the timm library. We calibrate both models using temperature scaling, setting the temperature of ResNet50 to T = 0.90 and T = 1.025 for VIT. For POEM specifically, we implement an action delay of 100 examples throughout the experiments this paper. This delay allows the monitoring component of POEM to accumulate sufficient evidence before updating the model parameters, mitigating the influence of potentially noisy initial data.
Data We randomly sample 25% of the examples from ImageNet validation set as an unlabelled holdout set. The corresponding corrupted examples are excluded from ImageNet-C maintain the validity of the holdout data. All methods are evaluated only on the remaining 75% samples. Each experiment repeated 10 times with different holdout data splits and data selections, which is particularly crucial for experiments with small subsets of examples, such as the continual shifts experiments. Specifically for ViT, we modify the default preprocessing transforms offered by SAR, to the one used by the model during its training (see htttps://buga:. www.gugggggggccoooooo co/timm/ rit_base__ ft in1k for the exact details). This adjustment is crucial to ensure proper estimation of the source CDF well ensure the model's performance on in-distribution data adheres to the one reported Hugging Face.
patch16 224..22220 augreg2_ aairreg22ik
in21k_
Code, hyperparameters, and learning scheme We use the SAR 43 repository (available at https:// gi thub. cwm/mr-ooooooo aggpa//ttggpaaaaAaaaa.. for the ImageNet and ImageNet-C experiments. To ensure consistency with prior works, we adopt the hyperparameters, optimizers, and procedures provided within the SAR repository. This ensures that all baseline methods as well as POEM are run with the exact same settings. We also compare our method with COTTA using the code provided by the authors, available at httpss
com/ qinenergy//nieeeeeee cotta In more detail:
For all methods except COTTA:
The learning rate (1) in Algorithm 2 calculation follows these formulas:
VIT: learning rate ( (0.D01) x batch size
64
batch size x 2 We use SGD optimizer with momentum of 0.9 for self-training.
* ResNet50: learning rate = (0.00025
64
We use = 0.40 log (1000) (denoted by Eg in 42 43
For COTTA:
We tune the learning-ran search for each model (ResNet and VIT), ranging from 0.001 .i where E [0.5, 1, 2, 4, 8). and select the best best-performing value for each model on the continual setting with corruption segment size of 1, 000. This process resulted in the following learning rate values.
* ViT learning rate = (0.001 x 4
64
* ResNet50 learning rate
0.001
x
We use Adam optimizer with 8 (0.9, 0.999) and weight decay for self-training, as employed in the original work.
batch size of is consistently used throughout all of the experiments, and, as mentioned in Appendix we use D 1.8 and 7 8./3 z 0.0722 for our monitoring algorithm (see Algorithm 1.. Lastly, since the empirical CDF in Algorithm 2 (line 6) is calculated from a finite set of data points, it inherently creates step function. In practice we use linear interpolation create a continuous function for better operation.
24

---

Hardware All experiments are conducted on our local server, equipped with 16 NVIDIA A40 GPU 49GB GPUs, 192 Intel (R) Xeon (R) Gold 6336Y CPUs, and ITB of RAM memory. Each experiment uses single GPU and 8 CPUs.
F.2.1
Additional experiments: continual shifts
ResNet(GN) Acc.
Method
TENT
COTTA
40
SAR
20
POEM (ours!
Gauss. Noise
Shot Noise
Impulse Noise
Detocus Blur
Glass Blur
Motion Blur
Zoom Blur
Snow
Frost
Fog
Bright.
Contrast
Elastic Trans
Pirel.
Juag Comp.
time
Corruption lype (Severity 5)
Res Acc.
36.0
32.0
30
30.0
2
3
2
1
2
100
500
1000
2000
Severity Level (Gaussian Noise)
Severity Level (Gaussian Noise)
Corruption Segment Size
time
time
Figure 5: Continual test-time adaptation on ImageNet-C with ResNet model. Top: Per- corruption accuracy with a corruption segment size 1,000 examples. Results are obtained over 10 independent trials; error bars are tiny. Bottom left: Severity sowft-low (1) to high (5) and back to low. Bottom center: Severity shift shift-high (5) to low (1) and back to high. Bottom right: Mean accuracy under continual corruptions function of the corruption segment size.
Supplementary details for the continual shifts experiments presented in the main manuscript For the corruption shift experiment, we used corruption segments sizes the range of 100, 250, 500, 1000, and 2000. Apart from the bottom right panels of Figures 2 and the results are obtained using corruption segment size of 1,000 exclusively.
Experiments with a ResNet (GN) model Herein, we repeat the same experiments from Section of the main manuscript to evaluate our proposed method within a continual shift setting, but now focus on a ResNet50 (GN) model. The results are summarized in Figure [3] showing similar trend to that obtained with the ViT model, albeit accuracies that are closer note that the baseline accuracy of ResNet is far below that of VIT. The bottom right panel of Figure 5 demonstrates POEM's efficiency adapting to fast-changing distributions with only a few examples. While the advantage of POEM is less clear with small corruption segments in ResNet, it becomes evident as early as segment size 500. Observe the bottom right panel of Figure 5 At corruption segment size of 2,000, POEM surpasses the best baseline method's accuracy (EATA) by 1.27%. POEM also achieves 4.64% increase compared to the original model (no adaptation).
baseline methods. We
F.2.2 Additional experiments: single shift
Table I summarizes baseline methods' accuracy ImageNet-C for ViT and ResNet models. Table 2 offers a more detailed breakdown of the results Table T, showing accuracy for each corruption type.
F.2.3 Additional experiments: in-distribution and the behavior of
Even though adapting unseen in-distribution data from ImageNet attains similar accuracies for all baseline methods compared to the no-adapt approach (Table POEM maintains this accuracy with the most minimal change to the original model's parameters and virtually unchanged ECE. This
25

---

Table 1: Single shift test-time adaptation. Accuracy evaluated on ImageNet-C, averaged over all 15 corruptions of severity level 5. Detailed results are in Table 2 We attribute COTTA's inferior performance to its learning rate being tuned for the continual setting; see Appendix R:2 for details.
ResNer50 (GN)
ViT
Method
(I.N)
No adapt
31.44
51.65
TENT
25.46
62.21
COTTA
23,90
55.31
EATA
38.63
64.14
SAR 36.01
64.03
POEN (ours)
38.90
67.36
Table 2: Summary of performance metrics of adaptation methods on the ImageNet-C dataset, evaluated for each type of corruption severity level 5. The results are evaluated 10 independent experiments, conducted for each method and corruption type; standard error is presented.
Gomptter
Caro
Imgalio
Daforas
Gloss
Marien
Zoom
Fosd
Feg
Corra
Bhare
22.31
55.0
35.96
18.65
15
1.
700
508
1672gos
332
00
55-78go
00
7201ge1
16:02a
EATA
20.22
1858
29
46.11.aa
47.99133
10.2
4600
28.18a3
48.0
SAR
31:37 am
M431
31
18.8
17.7941
7201.e1
1548g31
11:0ca:
Hann
50 90.
41.00
22.14
85.01
96.16
W9.4
56.8.11
No s0apt
43.75
00.27
50411
12.72
36.60
50.07
44.71
32.11
10.41
18.56
31
45.01
10.33
00.30
$7.20 57.30
50ano
58 07
S7.3
10.1
on (Tum:
51.85g
14
75
78.17am
61 16g5
79.16ar
VIT
ESTA
OZT7A
56.73
58,17
64 34
46.37aa
$1.12
17,50a
20.44xx
05.20uss
91.0....
79.39
17.30a
6108.::
0050..2
78,00.0
31.4ba
7262
55
66
03.20,aa
65.41iau
73.99
70.00as
is demonstrated in the left panels of Figure 3 and Figure 6 Specifically, as seen in Table 3
||
for POEM is merely 0.17, significantly lower than other methods on ResNet50 over 30 times less change than the closest baseline method. This minimal change demonstrates POEM's capability for controlled adaptation.
Figure 7 further demonstrates the "no-harm" effect of POEM under in-distribution test data. The left panel shows the empirical CDF of the entropy values F. of all baseline methods, indicating that these lead to overconfident predictions. This is in contrast to POEM, whose estimated target CDF closely aligns with the source distribution. Such minimal deviation from the source distribution aligns with our previous findings POEM tends to keep the model parameters intact when adaptation is unnecessary.
When applying the model to out-of-distribution data (right panel of Figure 7). the unadapted model appears slightly under-confident, as indicated by its corresponding CDF F', being lower than F. Here, POEM effectively restores the model's original confidence by adjusting its estimated CDF to closely match the source CDF. SAR achieves comparable results POEM, but through different strategy. It employs restart mechanism that acts as safeguard, activated when the entropy of the adapted model drops below specific exponential moving average (EMA) threshold.
F.2.4 Ablation study
We assess the impact of gmatch++ (10) compared to gmatch on adaptation accuracy through isolated analysis. Table 4 shows that our basic match loss gmatch improves the test accuracy over the no- no-adapt baseline, even without filtering and weighting. This underscores the core strength of our approach. The enhanced -match+- that incorporates sample filtering and weighting mechanisms further boosts performance.
F3 CIFAR1O-C and CIFAR100-C experiments
Data CIFARIO-C and CIFAR100-C datasets are extensions of the original CIFARIO and CIFARIO0
test sets. These datasets consist of 15 different corrupted versions of the original CIFAR test images, with each corruption applied at 5 severity levels, mirroring the structure of ImageNet-C. our experimental setup, we randomly select 25% of the examples from original CIFAR test set to create
26

---

Table Results of adaptation on in-distribution ImageNet data,
Top-1 acc.
Top-5 acc.
Empirical cal. err.
||
Model
Method
No adapt
79.95
94.94
0.02433..00
0.00
79.63
94. 94.78.0.02
0.0894g0.00
5. 5.90
ResNet50 (GN)
COTTA
79.87
94.83
0.0341 1:000
29.08 1..08.0..000
EATA
79.01
94.89
0.0709
10.53
a9.97
20.93
0.0250ga.om
8. 8.14go.01
POEM (ours)
79.95
94.93
0.02443:0.00
0.17
No adapt
84.5220.05
97.30
0.0099g0.00
0.00gg0.00
TENT
84.4220.04
97.30
0.0135
4.35
VIT (LN)
COTTA
84.47
97.35.0.0
0.0209
13.75
84.57
97.35
0.01141
6. 20.05
SAR
84.52g0.05
97. 97.32200.0
0.0101
0,86 2000
PORM (ours)
84.48
07.29
0.009820.00
0.32g..00
Param, change
10-
Mothod adapt
Betting variable
1.0
ditribution
Severity 1 (bright.) Severity 5 (bright.)
|o]
TENT
EATA
POEM tours)
0.0
0.025 0.050 0.075
5000
10000
15000
20000
25000
30000
35000
ECE
Example
Figure 6: In-distribution experiment on ImageNet (left panel); calibration error (ECE [10]]
versus ||w||ž-a metric that evaluates the classifier's parameters deviation from the original ResNet50 model. Lower values on both axes are better. Results are averaged across 10 independent trials; standard errors and accuracy each method are reported in Table in the appendix. The behavior of the betting parameter (right panel): the value of is presented as function of time for both in- and cut-of-diitribution experiments (a single shift, severity levels).
unlabelled holdout sets of sizes 2,500 images. To preserve the integrity of the holdout set, exclud the corresponding corrupted examples from CIFAR 10-C and CIFAR 100-C, respectively, All adaptation methods were applied on the remaining 75% of the data, ensuring consistency across approaches, regardless of their holdout set requirements. We conduct each experiment for 10 independent trials.
Model Our experiments utilize pre-trained ResNet32 model with batch normaliza- tion (BN), obtained from torch-hub and available httpp:://github.ccom//tt pytorch-cifar-modd
Methods, code, hyperparameters, and learning scheme The pre-trained ResNet32 architecture includes batch normalization (BN) layers, which forces us This differs from the batch-size we used in the ImageNet experiments, which was equal to In what follows, we compare POEM to SAR, EATA, and TENT only. We do not conduct experiments with COTTA, as our ImageNet experiments showed that COTTA performs inferiorly when the self-training batch size is small. To ensure fair comparison, perform grid search for choosing the optimal learning
batch-size of 4 during self-training.
Table 4: Ablation study: the effect of gmatch compared to gmatch++ on the accuracy of POEM. Results are presented for ImageNet-C and averaged over all 15 corruptions of severity level 5,
Method
1
ResNet50 (GN)
VIT (L.)
No adapt n/a
31.46
51.65
POEM
32.49
60.64
/
38.90
67.36
27

---

In-distribution
Single shift
adopt
Cumulative Prob.
TENT
SAR
0.4
0.0
Entropy
Entropy
Figure 7: Empirical test entropy CDF of each adaptation method, applied to in- and out-of- distribution ImageNet data. The dotted black line represents the source CDF F. obtained by applying the original ResNet50 model on test images from the source domain. The left panel shows how self- trainnng on the validation set of ImageNet (in-distribution data) affects the entropy distribution of the model. The right panel repeats the experiment on out-of cittof-distribution data from ImageNet-C with brightness corruption of severity level 1.
Comruption Accuracy
Learning Senotivity
Comvution Accrary 9. S
Sovetown Aaster
1797
A024 Jaunt
Mear
10
Bor
ceryon Rove
dightios
For
Soom tour
reot
popona Base Catrent
Iey Comcrestoo
Taar
Pita Soon
16:00
56:55
0.2001 00005
0.001
betene
diss
Leaming Rate
Type ISeveliy
Slowdown
Learning Sensibivtty
Mathrdd
Comuetion rocy
Stowepoen Botor
TENT
mar
crom
rod
inst
dies our
oate
Contant
Decarastion
Baction
2007
ECOH Shurd
640CL 88605 0003
Learning Rale
Compttion 3ys (eeeriy
Figure Short-term adaptation performance on test of about 15,360 samples from CIFAR10-C (top) and CIFAR100-C (bottom) using a ResNet (BN) model. Left: Continual test-time adaptation performance showing per-corruption accuracy with a corruption segment size of 1,024 examples of severity level 5. Results are averaged over 10 independent trials with error bars indicated. The adapt' baseline is not displayed its mean accuracy is significantly lower than other methods (see the right panel); we omitted this method to enhance visualization of differences between the adaptation techniques. Middle: Runtime slowdown, defined as the runtime of test-time adaptation divided by the runtime of the source (no-adapt) model; lower is better. Right: Learning rate sensitivity study, showing mean performance across the continual experiment. The results the left panel are obtained with the best learning rate for each method.
rate 17 for each method. The details of this sensitivity analysis are presented in the following section. In the case of POEM, we retained the hyperparameters of the monitoring tool as outlined in Section
E.3.1 Continual shifts experiments
Short-term adaptation performance We focus on the continual setting where the corruption type is changing over time, akin to our ImageNet experiments. Each corruption type includes 1, 024 samples, resulting in a test set of approximately summarized in Figure g Following that figure, one can that our method is competitive and even
1, 024 Rs 15, 000 samples. The results are
28

---

Coruuption Accuracy Over Time
Runtime Slowdown
Learning Rate Sensttvitt
Commento AcCrrte
Stondonn Pector
9 10
udupt
DAT
Souns)
&
Fir
Srow
Desccs Ale
drginune
Fig
Jaaes tha
rost
coes yoe
oppare Noom
lany Comprnsatio
Content
TEaT
Edil
0975 Learing Rate
POOM
Comuption 1yze ISeventty 3)
Corruption Accuracy Over Time
Runtime Slowdown
Learning Rate Sensitviity
Method
Comvetion recy
Stowtoen Factor
CAR
Mean
Foy
caan the satte Cantist kens compecztion
Blstie treetove
Eedt
Horm
Drince
PGT
Leaming Pats
Comuption 1ype (oevrriyy 5)
Figure 9: Long-term adaptation performance on a test set of 1 112,500 samples from CIFARIO-C (top) and CIFAR100-C (bottom) using ResNet (BN) model. Continual test-time adaptation is applied with a corruption segment size of 7,500 examples of severity level 5. The other details are as in Figure 8
outperforms baseline methods terms of accuracy. Runtime comparisons (relative to the no-adapt model) are also presented, demonstrating that our method's complexity is similar to TENT and EATA, and lower than SAR. Additionally, the sensitivity study for the learning rate parameter v reveals our method's robustness to this hyperparameter, particularly when compared to SAR and TENT.
Long-term adaptation performance Here, we conduct a similar experiment, but on a larger test set containing 112,500 corrupted samples (15 versions of 7,500 images). The results, summarized in Figure 9 show that our proposed method is competitive with the baseline methods in terms of adaptation accuracy. Notably, our method's 's runtime is twice as fast as SAR and comparable to EATA and TENT. Unlike SAR, we do not employ model-reset mechanism, nor do we use an anti-forgetting loss like EATA yet our approach demonstrates robustness over the long term. In contrast, the right panel of Figure reveals that TENT highly sensitive the choice of learning rate.
F.4 Office-Home experiments
Data The OfficeHome dataset consists of images from 4 domains: "Art", "Clipart". "Product", and "Real World". It contains a total of 15,588 images across 65 object categories. The "Art" domain has 2,427 images, "Clipart" has 4,365 images, "Product" has 4,439 images, and "Real World" has 4,357 images. We focus on adaptation from the "Real World" domain to the "Art", "Clipart", and "Product" domains. We chose this setup over continual setting as it is deemed more natural for this dataset. Given the lack of predefined data structure, we split the dataset into an 80% training set from the "Real World" samples, with the remainder serving validation and holdout sets for method and EATA.
Methods, model, hyperparameters, and learning scheme We fine-tune the last layer of the ResNet50 with Group Normalization (GN) previously used in the ImageNet experiments. We fit the model on 80% of the "Real World" examples for 25 epochs, with the best model saved based on performance on the remaining 20%. We use Adam optimizer with default Py PyTorch hyperparameters and set the learning rate to 5 10-5, Similar to the CIFAR experiments, we compare POEM to SAR, EATA, and TENT only; our ImageNet experiments showed that COTTA performs inferiorly when self-training batch size is 1, which is used in this experiment as well. Learning rates are tuned for each method using a predefined grid, ensuring a fair comparison, similar to our CIFAR experiments. The hyperparameters of POEM's monitoring tool specified in Section
29

---

Accuracy
Runtime Slowdown
Learning Rote Sensitvitty
Stoedow Pector
CATS
Souns!
10-
Pooket
CA6SS 3.999
3.211
0.966
Target Comann
Leaming ng Ra5a
Figure 10: Performance analysis of test-time adaptation methods on the Office-Home dataset using a ResNet (GN) model, pre-trained on ImageNet and fine-tuned on the "Real World" source domain. Left: Test-time accuracy for adaptation from the source domain to three distinct target domains ("Art", "Clipart", and "Product"). Results are evaluated on the complete test dataset of each target domain and averaged across independent trials, with error bars indicated. Middle: Runtime slowdown comparison. Right: Learning rate sensitivity study, displaying the average accuracy across all three target domains. The results the left panel are obtained using the best learning rate for each method.
F.4.1 Single domain adaptation experiments
Test-time adaptation is applied to the entire test data of each target domain ("Art", "Clipart" and "Product"). Results summarized in Figure TO Overall, all the methods demonstrate modest accuracy gains compared to the 'no-adapt" case. Our proposed method POEM slightly outperforms TENT and EATA in terms of accuracy, while achieving results comparable SAR. terms of compu- tational efficiency, our method's 's runtime is on par with TENT and EATA, and notably faster than SAR. Regarding sensitivity the choice of the learning rate, our approach displays superior robustness compared to TENT and SAR, and a similar robustness to that of EATA.
G Future Directions
In future work, we plan to complement our empirical findings with a theoretical analysis. Our goal is to rigorously determine when entropy matching is superior to entropy minimization, thereby uncovering the theoretical benefits and limitations of our approach.
Another future direction is to support POEM with the ability to handle label shift at test time. This challenge exemplified by scenarios where the source domain has balanced label distribution, but the test domain becomes unbalanced. In such cases, our current monitoring tool might detect this label shift and trigger unnecessary adaptation the absence of covariate shift. This underscores the need for monitoring tool that remains invariant to label shifts. To address this challenge, one may consider two potential approaches. The first builds on ideas from [133, particularly their vredicionn-bllnce reservoir sampling technique. This method can be used approximately simulate an i.i.d. data stream from a non-i.i.d. stream sensitivity to label shifts. The second approach may involve the use of a weighted source CDF instead of the standard source CDF, with weights corresponding to the likelihood ratio P'(Y)/P*(Y) This concept, borrowed from conformal prediction literature [114], aims make the test loss to "look exchangeable" with the source losses, thus adjusting for label shift. The main challenge here lies in reasonably approximating the likelihood ratio Pt PY(Y)/P(YY) especially when facing simultaneous covariate and label shifts at test time. The ideas presented 115] may offer a promising starting point for exploring this avenue.
manner, potentially reducing our martingale process's
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We claimed to have contributed the following contributions:
We develop a sequential for classification entropy drift detection: see Section 3.3
Appendix Appendix [ and Appendix E
We show how to utilize the test martingale to analytically design mapping function that transports the classifier entropies obtained at test time to resemble those of the source domain. The derivation of the algorithm is given in Sections 3.4 3.5
and
Appendix(E
We also conducted wide range of experiments to support our approach in Sections 3.2 and Appendix
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We describe the problem setup and our assumptions in Section 2. and further diseuss the limitations of our work Section 5
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The theoretical claims provided Section are proved in Appendix
3.3 Appendix and Section 3.4
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In Appendix Fewe provide all of the implementation details. The code used conduct the experiments attached to the submission as supplementary material. An open-source GitHub repository would be published upon publication.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: ImageNet and ImageNetC are both publicly available datasets. The data used in the synthetic experiment can be reproduced by running the code provided or implementing it based on Appendix [F] All the results reported in the paper can be reproduced by running our software package.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The experiments settings are described in Section 3.2 and Section All the implementation details are extensively discussed in Appendix
31

---

7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We show the standard emror in all graphs; also report the standard errors in Tables and 3
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix F.2 we detail the computational resources we used to conduct the experiments in this work.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://ppw.....a.......////////a
Answer: [Yes]
Justification: The work is done in appropriation with http:://nourrpp..ccpbbb.. Cehiidesneelis
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the broader impacts of this work in Section
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: We do not release anything that warrants a safeguard.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We credit prior contributions that we build upon, as well as provide links to the relevant code repositories. The open- source ImageNet and ImageNet-C datasets are also credited.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: While we do not provide new assets, we support the paper with a properly documented software package.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
32

---

Answer: [NA]
Justification: No human subjects were used in this work.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects were used in this work.
33


Reviews:
[{"review_id": "Y7zXies1ch", "reviewer": "Reviewer_Utu5", "summary": "The paper addresses the problem of adapting a classifier to a new domain at test-time. It proposes a framework that first detects a distribution shift and, based on the detection results, adapts the classifier. The distribution shift detector employs a sequential test evaluating if the distribution of the test entropy deviates from the distribution of the source entropy. In the adaptation step, the normalization parameters of the model are updated via an entropy matching objective. Experiments are conducted on ImageNet-C.", "questions": "- I’d appreciate some more clarification regarding section 3.4 (adaptation mechanism). Could you please elaborate on the role and interpretation of $Q$. I understood from lines 260-264 that $Q$ can be thought of as the distribution that approximates the unknown target’s entropy CDF, which makes sense to me given equations (3) and (6). However, in line 266 $Q$ seems to be used as a function to transform $u_j$ to $\\tilde{u}_j$ by $\\tilde{u_j} = Q(u_j)$. Could you explain the link between $Q$ being the target’s entropy CDF as well as a transformation?\n\n- I think the entropy matching could potentially be a good alternative to the dominant paradigm of entropy minimization, particularly since the latter has been shown to collapse eventually [1]. How does entropy matching perform on long-range adaptation? My understanding from the experiments is that only 1000 samples per corruption are used and not even the entire ImageNet-C dataset.\n\n[1] Press, Ori, et al. \"The entropy enigma: Success and failure of entropy minimization.\" arXiv preprint arXiv:2405.05012 (2024).", "limitations": "The limitations are rather vague, and it seems currently unclear which limitations the method encounters.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The idea of using test martingales for test-time adaptation is novel to my knowledge and seems like a very natural application and interesting research direction.\n- The paper claims it only adapts when the distribution actually shifts, which is a desirable property of a TTA method, leveraging the trade-off between agile adaptation and keeping the source knowledge.\n- The paper is very well written overall. The reader is provided with intuitive explanations of the testing-by-betting framework - a very technical and not easy-to-explain theory - which makes the paper pleasant and insightful to read.\n- The experiments on ImageNet-C show encouraging results.", "weaknesses": "**In short:** The paper proposes a very interesting approach, but it seems to me more work is required to round out the paper. In particular, the paper requires more empirical validation and clarifications.\n\nSome of the main weaknesses I see are:\n\n- The experimental results are quite limited. By showing results on only one dataset against three baselines, it is a bit unclear how the method performs across different datasets compared to existing methods. Comparing on other standard TTA benchmarks (such as CIFAR-10-C, CIFAR-100-C for corruptions, or Office-Home for domain adaptation) could help determine in which settings the method provides most gains and also its limitations.\n- The paper would benefit from an illustration visualizing the entropy matching procedure. In particular, it would be helpful to illustrate how $u_j, \\tilde{u_j}, Z^t_j$, and $\\tilde{Z}_j^t$ connect via the functions $F_s$ and $Q$.\n- More space and explanation could be dedicated to the actual entropy matching procedure. How can we match the two entropy distributions given $u_j$? This seems currently concentrated in lines 265-271 (see questions below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for your valuable feedback and suggestions. We are glad that the reviewer found our approach to be novel and an interesting research direction. It is gratifying to see that the reviewer thinks that “the entropy matching could potentially be a good alternative to the dominant paradigm of entropy minimization.” Additionally, we appreciate the positive feedback regarding the clarity of our writing and that the reviewer found the experiments on ImageNet-C show encouraging results. Thank you!\n\n> The experimental results are quite limited\n\nKindly refer to the global response to all reviewers.\n\n> An illustration visualizing the entropy-matching procedure\n\nThank you for this great suggestion, we will include such an illustration in the revised paper. We also discuss the role of each component below.\n\n> Clarification on the entropy matching procedure. What is the role and interpretation of $Q$? How can we match the entropy distributions?\n\nThank you for raising this question, as it touches on one of the more nuanced aspects of our work. Indeed, as the reviewer mentioned, one can interpret $Q$ as the distribution approximating the unknown CDF of the target entropy $Z_t$. We understand that this might be confusing as $Q$ is a function of $u$. However, recall that $u$ is a function of $Z_t$, as $u := F_s(Z_t)$. Observe also that $Z_t = F_s^{-1}(u)$.\n\nTo better clarify the role of $Q$, consider a case where the betting is optimal in the sense of Proposition 2. The right-hand side of Eq. (8) gives the explicit form of the ideal $Q$ being $\\tilde{u} = Q(u) = F_t(F_s^{-1}(u))$. In turn, $Q(u) = F_t(Z_t)$. In practice, the test entropy CDF $F_t$ is unknown, yet the above relation highlights why the $Q$ we formulate via the betting function can be intuitively viewed as the distribution approximating the unknown target entropies CDF. This is due to the fact that any valid betting martingale is a likelihood ratio process, aligning with Eq. (8).\n\nAs for the matching property, observe that in the ideal case, the pseudo-entropy value $\\tilde{Z_t} = F_s^{-1}( \\tilde{u}) = F_s^{-1}(Q(u)) = F_s^{-1}(F_t(Z_t))$. This argument reveals the tight relation between our adaptation scheme and optimal transport: in the ideal case, the pseudo-entropy $\\tilde{Z_t}$ is obtained by applying the optimal transport map from the target entropy distribution to the source entropy distribution. Our experiment in Figure 6 in the appendix demonstrates that we indeed achieve distribution matching via the online-estimated $Q(u)$ in practice, although we do not have access to $F_t$ that varies over time.\n\nWe will include this discussion and clarification around Proposition 2 in the text.\n\n> How does entropy matching perform on long-range adaptation? My understanding from the experiments is that only 1000 samples per corruption are used and not even the entire ImageNet-C dataset.\n\nThe experiments we conducted also involved a long-range adaptation on a test set of size $\\approx 30,000$ for ImageNet; $\\approx 15,000$ for CIFAR; and for OfficeHome we used the entire test data. Focusing on ImageNet-C, we highlight that in the continual setup, we also used $2000$ samples per corruption (Figure 2 bottom right panel), resulting in a test set of size $2000 \\cdot 15=30,000$ samples in total. In the single corruption experiments, we use $37,500$ test samples for each corruption. Notably, we had to reserve a subset of the test set to implement both EATA and our method as the two need access to unlabeled holdout data from the source domain.\n\n> The limitations are rather vague, and it seems currently unclear which limitations the method encounters\n\nThe key limitations are the following:\n\n1. To implement our method, we assume access to the source CDF, evaluated on holdout unlabeled samples from the source domain.\n2. The choice of hyper-parameters, in particular the learning rate, can be challenging as it depends on the data and model used. This is akin to related TTA methods.\n3. The lack of theory that reveals when entropy matching is guaranteed to improve performance. This issue is one of our future research directions.\n\nWe will update accordingly the discussion on the limitations in the text. We thank the reviewer for this point.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HTKJwLdHTe", "reviewer": "Reviewer_Nyvv", "summary": "This paper introduces a novel method for test-time domain adaptation using online self-training. It combines a statistical framework for detecting distribution shifts with an online adaptation mechanism to dynamically update the classifier's parameters. The approach, grounded in concepts from betting martingales and optimal transport, aligns test entropy values with those of the source domain, outperforming traditional entropy minimization methods. Experimental results demonstrate improved accuracy and calibration under distribution shifts.", "questions": "1. Can you explain how betting and martingales specifically contribute to the modeling of entropy CDF and why you believe this is a suitable approach for TTA?\n\n2. In Figure 1, the comparison between entropy minimization and entropy matching is somewhat unclear. Can you elaborate on the intended interpretation of this figure and how it supports your claims about optimization following either the black or red lines?\n\n3. The requirement to calculate the source CDF adds extra computational overhead and potential privacy concerns. How do you propose mitigating these issues, especially in scenarios where source model access is restricted or where computational resources are limited?\n\n\n\n\n---", "limitations": "1. The paper uses the term \"domain adaptation,\" which traditionally implies multiple epochs of adaptation and separate target training and testing samples. In the context of test-time adaptation (TTA), this terminology may cause confusion. A clearer distinction between these methodologies is suggested to avoid misinterpretation.\n\n2. The requirement to calculate the source CDF introduces additional computational overhead, which has not been thoroughly discussed. In practical scenarios, especially where computational resources are limited or where source model access is restricted due to privacy concerns, this could pose significant challenges. An analysis of the method's computational efficiency and potential solutions to mitigate these issues would be beneficial.\n\n3. The experiments conducted do not include widely recognized TTA datasets such as CIFAR10-C, CIFAR100-C, or domain adaptation datasets like OfficeHome and DomainNet. Including these datasets in the evaluation would provide a more comprehensive understanding of the method's generalizability and robustness.\n\n4. The paper does not compare its results with state-of-the-art TTA baselines.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It is interesting to see betting and martingale appear in test-time adaptation, especially for modeling CDF for better shifted test sample prediction.\n2. The paper is overall easy to follow, with rich experiments and visualizations. The algorithms clearly explain how the framework works.\n3. The experiments on two TTA settings (single domain and continual TTA) confirm its effectiveness.", "weaknesses": "1. It is not very intuitive to use betting in TTA. Although it might work for modeling CDF, martingale itself is not naturally suitable for TTA entropy CDF.\n2. It is not very suitable to use the term \"domain adaptation\" in this context. Domain adaptation typically allows multiple epochs for adaptation, even in source-free settings, and uses target training samples while evaluating on target testing samples. In TTA, the same set of test data is used for adaptation and testing.\n3. In Figure 1, the comparison with entropy minimization shows peaks, while entropy matching shows valleys when facing data in the tail for both classes, indicating they are both good indicators for class boundaries regardless of the changed data distribution. However, as described in lines 176-181, it seems the optimization will follow either the black line (entropy minimization) or the red line (entropy matching), making the meaning of this figure a bit vague.\n4. The model requires calculating the source CDF, which requires extra time. Additionally, if there are source privacy concerns, it may not be possible to perform such calculations if the source model is only made available.\n5. The experiments do not include an efficiency study, which is one of the motivations for doing TTA.\n6. The paper does not include commonly used TTA datasets such as CIFAR10-C, CIFAR100-C, or any domain adaptation datasets such as OfficeHome, DomainNet, etc.\n7. There is no sensitivity study.\n8. This paper does not compare with state-of-the-art TTA baselines such as ROID [1].\n\n[1] Marsden, R. A., Döbler, M., & Yang, B. (2024). Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2555-2565).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and the constructive review. We are pleased that the reviewer appreciates the novelty and clarity of our writing. We very much value the reviewer’s comment that “the experiments on two TTA settings (single domain and continual TTA) confirm its effectiveness.” Thank you!\n\n> Why is betting martingale a suitable approach for TTA?\n\nA betting martingale is a powerful tool to monitor distribution shifts in an online manner. We designed a monitoring tool that tests whether the distribution of the classifier’s entropy values drifts at test time. This naturally leads us to ask: if a martingale can detect that the model “behaves” differently at test time, why not use this knowledge to correct the model’s behavior?\n\nOur work shows how the evidence for distribution drift encapsulated in the betting martingale can be used to adapt the model at test time. The idea is to use the martingale to transform the test entropies to “look like” the source entropies—essentially matching the distribution of the source and the self-trained model entropy distributions. This, in turn, builds invariance to distribution drifts, which is a key principle to improve model robustness.\n\nIn the interest of space, for a more technical reply, we kindly refer the reviewer to the response we provided to Reviewer Utu5 (“Clarification on the entropy matching procedure”).\n\n> The use of the term \"domain adaptation\". \n\nYour point is well taken! If given the opportunity, we will fix this issue in the revised paper and use “test-time adaptation” instead. We will also remove the word “Adaptation” from the title of the paper. Thank you for your constructive comment.\n\n> Can you elaborate on the intended interpretation of Figure 1?\n\nEach curve in Figure 1 presents a different risk (black for entropy minimization, red for entropy matching) as a function of the weight $w$ of the classifier $f_w$. Since the optimization procedure aims to minimize a given risk function by changing the value of the weight $w$, the curves help to understand what is the optimal value that should be obtained.\n\nBy minimizing the entropy risk, the optimization ends with a self-trained classifier $f_w$ that achieves the smallest value of the black curve. This results in a trivial classifier that always predicts $+1$ (or always $-1$), regardless of the value of $X$.\n\nBy minimizing the entropy matching risk, the optimization ends with $f_w$ that achieves the smallest value of the red curve. This results in a classifier that separates the two classes as much as possible. Indeed, our online method obtained a self-trained classifier whose accuracy (nearly) matches the accuracy of the Bayes optimal classifier both under an in-distribution setting (top panel) and under an out-of-distribution setting (bottom panel).\n\n>The model requires calculating the source CDF, which requires extra time. \n\nRecall that this is a CDF of 1-dimensional variables (the source entropies), which is computed only once and offline. The complexity of computing this CDF is dominated by the evaluation of the pre-trained model on relatively small holdout unlabeled samples from the source domain. At test time, we only need to compute the value of the pre-computed CDF at a single point (the test point’s entropy value), which amounts to accessing a small, pre-computed 1-dimensional array.\n\n> What if computational resources are limited?\n\nFollowing the global response to all reviewers, the new experiments show that the runtime of our method is comparable to TENT and EATA and even lower than that of SAR.  Moreover, our monitoring tool can be used to decide whether the model should be updated or not at test time (that can further reduce runtime), as the martingale process detects distribution shifts. Notably, our monitoring tool can be applied in a “black-box” manner as it only requires access to the output of the softmax layer.\n\n> Additionally, it may not be possible to perform such calculations if the source model is only made available.\n\nIndeed, we require access to a pre-trained source model and a *pre-computed* source CDF. Importantly, given the two, we do not require any additional access to samples from the source data at test time. In that respect, our work does not differ significantly from EATA, which also assumes access to holdout source examples.\n\n> Limited evaluation; there is no sensitivity study.\n\nKindly refer to the global response to all reviewers.\n\n> This paper does not compare with state-of-the-art TTA baselines such as ROID\n\nOur goal is to highlight why we believe it is important to transition from entropy minimization to online entropy matching. Therefore, we deliberately compared our approach to strong baseline methods that are based on entropy minimization.\n\nThe ROID method builds on a weighted version of the soft likelihood ratio loss as a self-supervised loss. This loss departs from the line of the baseline, entropy-minimization methods we focus on, but it opens an interesting future direction. Broadly speaking, our paper offers an online mechanism for matching source and target distributions of any given self-supervised loss. In the context of ROID, it will be illuminating to explore how our matching paradigm would perform in combination with the weighted soft likelihood ratio loss instead of the entropy loss. We will include this idea for future work in the text!\n\nMoreover, ROID highlights that self-training can fail to improve or even deteriorate performance. This motivated the authors of ROID to include several, complex components within the test-time training scheme. Naturally, a SOTA method would include various components to enhance performance, and this set of ideas can be also valuable for our method. However, given that we introduce a new concept to the ever-growing test-time adaptation literature, we believe such explorations go beyond the scope of the current paper. These will divert attention away from our central message.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YKenE51xJx", "reviewer": "Reviewer_3rQG", "summary": "This paper proposes a novel test-time adaptation method based on martingales and online learning. It detects whether testing samples need to be adapted based on the sequential entropy values. Then, if a sample needs to be adapted, a pseudo-entropy value is computed for the adaptation. Overall, the idea of this paper is reasonable and interesting.", "questions": "Please refer to the `Weaknesses` section. \n1. The experiments show that the proposed method can achieve a lower ECE value. A further discussion about how these results can be done and their benefit for practice.", "limitations": "The authors have discussed the limitations at the end of this paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors replace entropy minimization with entropy matching, which is interesting. Under this main idea, online drift detection and online model adaptation are naturally proposed and make sense.\n2. This paper is well written and easy-to-follow.", "weaknesses": "1. The experiments are relatively weak as the authors only conduct experiments on the ImageNet-C dataset, ignoring the CIFAR10-C and CIFAR100-C datasets. It would be better to present the results on datasets with a small number of classes.\n2. The proposed method estimated the pseudo-entropy at testing time. However, I wonder whether this can be done when the label distribution [1, 2] also shifts because the shifted label distribution also affects the sequential entropy values. This should be discussed in detail, as well as the related papers.\n3. The “Protected” in the title of this paper and name of this method should be carefully considered as the overall method seems not to explicitly ensure the safety of performance.\n\n[1] NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation. NeurIPS 2022\n\n[2] ODS: Test-Time Adaptation in the Presence of Open-World Data Shift. ICML 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for your positive feedback and valuable suggestions. We are pleased that the reviewer found that “online drift detection and online model adaptation are naturally proposed and make sense.” We appreciate the positive feedback regarding the clarity of our writing. Thank you!\n\n> The experiments are relatively weak as the authors only conduct experiments on the ImageNet-C dataset, ignoring the CIFAR10-C and CIFAR100-C datasets.\n\nKindly refer to the global response to all reviewers.\n\n> The proposed method estimated the pseudo-entropy at testing time. However, I wonder whether this can be done when the label distribution [1, 2] also shifts because the shifted label distribution also affects the sequential entropy values.\n\nThank you for raising this important issue. Extending our proposal to the label shift setting remains an open question for us. The idea in [1] may serve as a promising starting point. Specifically, we found the idea of prediction-balanced reservoir sampling appealing, as it can be used to approximately simulate an i.i.d. data stream from non-i.i.d. stream in a class-balanced manner. This can potentially reduce the sensitivity of the martingale process to label shifts. In turn, we anticipate that under label shift and in the absence of covariate shift, our adaptation would be minimal; and in the presence of the latter, the adaptation would be more substantial, as desired.\n\nAnother possible approach would be to work with a weighted source CDF rather than the vanilla source CDF, where the weights should correspond to the likelihood ratio $P_t(Y)/P_s(Y)$. The use of such a weighted CDF was suggested in the conformal prediction literature to adjust for label shift between the source holdout data and test points [3], making the test loss to “look exchangeable” with the source losses. Certainly, the challenge in this context is to approximate the likelihood ratio $P_t(Y)/P_s(Y)$ in a reasonable manner. This challenge becomes more pronounced when faced with both a covariate and label shift. It may be the case that reference [2] pointed out by the reviewer could be a valuable starting point for us to explore this path.\n\nWe will include a discussion on this matter in the revised version of the paper. \n\n[1] T. Gong, et al. \"NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation,\" NeurIPS, 2022.\n\n[2] Z. Zhou, et al. \"ODS: Test-Time Adaptation in the Presence of Open-World Data Shift,\" ICML, 2023\n\n[3] A. Podkopaev and A. Ramdas, “Distribution-free uncertainty quantification for classification under label shift,” Uncertainty in artificial intelligence, 2021.\n\n> The “Protected” in the title of this paper and name of this method should be carefully considered.\n\nWe are now seriously considering removing the word \"protected\" from the title. We initially used this word for three main reasons. First, our monitoring tool rigorously alerts for distribution shifts, and the ability to raise such a warning is crucial for communicating with users that the model is encountering new environments. Second, our approach has been shown to have no harmful effect when the test data follows the same distribution as the source domain, which is a significant concern in test-time adaptation. Third, we wanted to acknowledge the protected regression method [1] that inspired our proposal.\n\n[1] Vladimir Vovk, “Protected probabilistic regression,” technical report, 2021.\n\n> The experiments show that the proposed method can achieve a lower ECE value.\n\nRecall that the ECE is presented when applying self-training on in-distribution test data (Figure 3, left panel). This experiment demonstrates that in this in-distribution scenario, our method maintains the ECE of the source model—we are not improving the ECE of the source model. Importantly, this stands in contrast to entropy minimization methods that, by design, drive the model to make over-confident predictions, as reflected by the increased ECE value in Figure 3. This emphasizes that when the test samples do not shift, we preserve the calibration property of the model and avoid making over-confident predictions, which is desired in practice. We will clarify the exposition surrounding Figure 3 in the text accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the reviewers' engagement with our paper and their valuable comments and suggestions. We will integrate their feedback into the revised paper and have conducted a new set of experiments, detailed below.

The reviewers acknowledged that the paper is well-written and introduces a novel approach for test-time adaptation.

The main criticism raised was that the experiments, though conducted on ImageNet-C, are quite limited. To address this, we have conducted additional experiments on CIFAR10-C, CIFAR100-C, and Office-Home datasets. In short, the new experiments show that **our approach is competitive with strong baseline test-time adaptation methods that are based on entropy minimization (TENT, SAR, and EATA)**. This conclusion aligns with the ImageNet-C experiments presented in the paper. 

### **CIFAR-10C and CIFAR-100C experiments**

We focus on the continual setting where the corruption type is changing over time. Using a pre-trained ResNet32 model, we applied online self-training with a batch size of 4 (due to batch-normalization layers). Each corruption type had $1024$ samples, resulting in a test set of approximately $15 \cdot 1024 = 15,000$ samples. To ensure a fair comparison, we tuned the learning rate for each method using a pre-specified grid; see the sensitivity study below. Notably, we did not change the hyper-parameters of the monitoring tool and used the same values as those employed in our ImageNet-C experiments.

Results are summarized in Figure 1 (attached PDF):

* Our method is competitive and often outperforms baseline methods in terms of accuracy. 
* Runtime comparisons (relative to the no-adapt model) are also presented, demonstrating that our method's complexity is similar to TENT and EATA, and lower than SAR.
* A sensitivity study for the learning rate parameter reveals our method’s robustness to this choice, particularly when compared to SAR and TENT.

### **Office-Home experiments**

We focused on adaptation from the “Real World” domain to the “Art”, “Clipart”, and “Product” domains. A continual setting was deemed less natural here. We fine-tuned the last layer of a ResNet50 model with group-norm layers (pre-trained on ImageNet1K) on Office Home's real-world images, reserving 20% as a holdout set for our method and EATA. Test-time adaptation was applied to the entire test data of each target domain. Learning rates were tuned for fair comparison, similar to the CIFAR experiments.  For consistency, we kept the same hyper-parameters for the monitoring tool as those used in our ImageNet-C experiments. As such, the same hyper-parameters for the monitoring tool are used across all experiments, regardless of the model or dataset.

Results are summarized in Figure 2 (attached PDF):

* Overall, all the methods demonstrated modest accuracy gains compared to the ‘no-adapt’ case. Our proposed method slightly outperformed TENT and EATA in terms of accuracy, while achieving results comparable to SAR.
* In terms of computational efficiency, our method's runtime was on par with TENT and EATA, and notably faster than SAR.
* Regarding sensitivity to the choice of the learning rate, our approach displayed superior robustness compared to TENT and SAR, and a similar robustness to that of EATA.


We provide individual replies to specific comments from each reviewer.

Meta-review: This paper presents a test-time adaptation (TTA) method which detects distribution shifts at test time as well as proposed a scheme to adapt the model to such shifts.

The paper received generally positive reviews with scores of 6,5,5.

In the original reviews, the reviewers had expressed some concerns and had requested for some additional experiments. The authors have also reported additional experiments in the rebuttal (e.g., long range adaptation experiments) and said that they will include it in the revised version.

The paper does have both a theoretically sound TTA method as well as pretty reasonable experimental results. Given the generally positive reviews and my own assessment from reading the paper, I think the paper makes good contributions to the area of TTA. Therefore, I recommend the paper for acceptance.,
Decision: Accept (poster)