The Over-Certainty Phenomenon in Modern Test-Time Adaptation Algorithms
Authors: Fin Amin
Jung-Eun Kim
Venue: Accepted by TMLR ()

Abstract: When neural networks are confronted with unfamiliar data that deviate from their training set, this signifies a domain shift. While these networks output predictions on their inputs, they typically fail to account for their level of familiarity with these novel observations. Prevailing works navigate test-time adaptation with the goal of curtailing model entropy, yet they unintentionally produce models that struggle with sub-optimal calibration—a dilemma we term the over-certainty phenomenon. This over-certainty in predictions can be particularly dangerous in the setting of domain shifts, as it may lead to misplaced trust. In this paper, we propose a solution that not only maintains accuracy but also addresses calibration by mitigating the over-certainty phenomenon. To do this, we introduce a certainty regularizer that dynamically adjusts pseudo-label confidence by accounting for both backbone entropy and logit norm. Our method achieves state-of-the-art performance in terms of Expected Calibration Error and Negative Log Likelihood, all while maintaining parity in accuracy.

Published in Transactions on Machine Learning Research (07/2025)
The Over-Certainty Phenomenon in Modern Test- Time Adaptation Algorithms
Fin Amin
samin2@mcsu.edi
Deportment of Electrical and Computer Engineering North Carolina State University
Jung-Eun Kinj Department of Computer Science North Carolina State University
Reviewed on OpenReview: opperevvee. net/ met/pdf? id= AGQRi
Abstract
When neural networks are confronted with unfamiliar data that deviate from their training set, this signifies domain shift. While these networks output predictions their inputs, they typically fail to account for their level of familiarity with these novel observations. Prevailing works navigate test-time adaptation with the goal of curtailing model entropy, yet they unintentionally produce models that struggle with sub-optimal calibration a dilemma we term the over-certainty phenomenon. This over-certainty in prodictions can be particularly dangerous in the selting of domain shifts, as may lead to misplaced trust. In this paper, we propose a solution that not only maintains accuracy but also addresses calibration by mitigating the over-certainty phenomenon. To do this, we introduce certainty regularizer that dynamically adjusts pseudo-label confidence by accounting for both backbone entropy and logit norm. Our method achieves state-of-the-art performance in terms of Expected Calibration Error and Negative Log Likelihood, all while maintaining parity in accuracy.
1 Introduction
When encountering new enviromments, humans naturally adopt cautious approach, assimilating the novelty to guide their decision-making This inherent ability to assess unfamiliarity and adjust certainty has not been entirely emulated in artificial neural networks. Unlike humans who might exhibit hesitation in unknown situations, many test-time adaptation (TTA) algorithms lack an explicit mochanism to modulate certainty in response to the novelty or unfamiliarity of their inputs.
Deep learning has never been a stranger the challenges of uncertainty. Over the past few years, the miscalibration problem of modern neural networks has gained substantial attention, highlighted by works such Guo et al. (2017), Abdar et. al. (2021) Liang al. (2017), and Paampari & Ermon (2020). However, how TTA algorithms themselves alter calibration is understudied. In this paper, we uncover the over-certainty phenomenon, a phenomenon that plagues many modern TTA algorithms by harming model calibration.
A prevailing strategy among TTA algorithms is the minimization entropy, either as an explicit target or as an inherent by-product of their methodology. And while this might bolster accuracy metrics, our rescarch indicates concerning trend: excessive entropy reduction can be detrimental to model calibration. What makes this trend more problematic is that it occurs within the context of a new domain, where epistemic uncertainty should typically be greater. In our work, we measure certainty the inverse of Shannon entropy of model's output after a softmax operation: H(f(z)) = (Eatropy(SootNax(((frr
*Corresponding author

---

Published in Transactions on Machine Learning Research (07/202) 2025)
1.00
ECE (lower is better)
Entropy (bils)
0.75
0.50
0.25
0.1
0.00
0.0
No Adoptation
SoTTA
TENT
ETA
T3A
Figure 1: This chart shows four modern TTA algorithms adapting EfficientNet to the clipart domain. Minimizing entropy is a common objective in recent work. However, this can have consequences on model calibration.
To further frame our discussion, TTA is used when model, trained source domain (X,; Y,), is presented with the challenges of different yet analogous target domain (Xe) with labels. In our problem setting, we do not assume access to X₁ until we adapt. The nuances between those domains, commonly termed domain shift, can introduce significant disruptions in model performance. TTA, in its essence, aspires to adapt the insights harvested from the source domain and apply them proficiently to the target domain, bypassing the need for labeled data in the latter. TTA is difficult because models must generalize to unfamiliar distributions without access to labeled data or strong assumptions about how the target domain relates to the source Fang et al. ) David et. al. (2010). Furthermore, as will elaborate in the next section it can be especially problematic to be too certain within the context of domain shift.
With the purpose of addressing these intertwined challenges, we introduce Dynamic Entropy Control. This TTA technique seeks to augment. accuracy and improve model calibration. By interweaving calibration inlo the core learning process, we produce a TTA adjustment algorithm that jointly improves accuracy while managing epistemic uncertainty. To summarize, our contributions arc:
The identification of the over-certainty phenomenon. We emphasize that our work not about the well-studied tendency of models to become less calibrated under distribution shift Pampari & Ermon (2020); Ovadia et al. (2019). Instead, show how recent trend in the design of TTA algorithms can yield models that are even more miscalibrated than the unadapted baseline. To the best of our knowledge, no prior work- work-including Press et al. (202)) -explicillyy analyzes this pattern while also providing thorough empirical evidence and mechanistic insight into why over-certainty emorges during adaptation.
Dynamic Endropy Control, a new TTA algorithm that achieves SOTA calibration in all of the four datasets and SOTA accuracy uplifts in the majority of domain shifts.
Our study is scoped to test-time adaptation algorithms that follow the prevailing trend of minimizing entropy on unlabeled observations in the classification setting. While we acknowledge the existence of TTA methods that do not follow this paradigm (e.g., Liang et al. 22000 : Tang et al. (2020)), they fall outside the focus of our analysis. Unless otherwise stated, all mentions of "TTA algorithms" in this work refer specifically to cntrooy-redduction--aaeddd approaches.
1.1 Why Care About Over-Certainty?
We emphasize that calibrating uncertainty estimates has ramifications that go well beyond quantitative metrics. For instance, in real-world scenarios that rely on trustworthy estimates of model confidence (e.g., autonomous driving, active learning, anomaly detection), an overconfident model can lead harmful misjudgments You et al. (2022). As a concrete illustration, consider a self-driving car that uses classifier to detect whether a neighboring lane is occupied:
2

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Pre-adaptation (low certainty): The vehicle's classifier might cautiously predict "safe to switch lanes" but at a low confidence. This low certainty prevents the car from making risky maneuver.
Post-adaptation (artificially high certainty): After applying traditional TTA algorithm, the classifier might become overly certain about its "safe to switch lanes" prediction. In reality, if this confidence is misplaced, the system can make high-stakes error that jeopardizes safety.
Thus, our approach seeks to preserve improve classification accuracy and produce calibrated uncertainty estimales, ensuring that decisions made based confidence thresholds are more reliable. Such reliability is also valuable in other domains that hinge on calibrated outputs, such as fairness-sonsitive applications Héberrt-Jhhsson et al. (2018); Creager et al. (022)) where miscalibration can exacerbate biases or distort outcome distributions.
Fairness Considerations. As suggested in Pleiss al. (2017), there growing recognition of how calibration can influence fairness. Overconfident models may disadvantage certain groups when high- confidence predictions guide resource allocation risk assessments. Ensuring well-calibrated probabilities is therefore crucial but often overlooked in current TTA approaches.
2 Related Work
Our literature survey covers methodologies catered towards updating a neural network on unlabeled data. For the sake of brevity, we will refer to unlabeled data as "observations." This section gives an overview the work done to improve networks on the fly. We start by introducing earlier work, such as dictionary learning techniques and lead our way into recent developments. We elaborate the algorithms we compare with in detail to give mechanistic insight why over-certainty happens. The next section covers how calibration is measured and improved in the context of domain shift. Lastly, we discuss assossing the reliability of an observation. Purthermore, we provide supplementary discussion of related work in Appendix A.3
2.1 The Test-Time Entropy Reduction Paradigm
The phrase "self-taught learning" was coined by RRina et al. (2007). In this work, the authors utilize obscrvations to find an optimal sparse representation of said observations. This sparse representation is used to train their model in lieu of the ordinary training set to improve out-of-distribution (OOD) performance.
Work in this field has extended a variety of approaches such the use of pseudo-labeling exploit the existing model's predictions target labels Lee et al. (2013); Mancini et al. (2018); Wang et al. 2022
be thought of under the guise of knowledge distillation (KD) Hinton et al (2015) Gou et al. (021). KD is a transfer learning paradigm where a large neural network, known as the teacher, transfers knowledgee to a smaller "student" network. Succinctly, the student is trained to match the output of the
Pseudo-labeling
teacher when given the same input as the teacher Stanton et al. (002)) In pseudo-labeling, the teacher and student are the same network.
The TENT algorithm introduces the trend of test-time entropy minimization Wang al. (2020a) Entropy minimization be thought. of using pseudo labels with the cross entropy loss function. In other words, the entropy minimization works by using gradient descent to minimize:
LTENT = Σ [ular) log (()lar)
(1)
VEC
to update the model's batch-normallization parameters.
More recent advancements this trend include EATA/ETA Niu et al. 2022 T3A Iwasawa & Matsuo 2021 and SOTAA Gong et al. (2024). ETA advances on TENT by making sure that observations are reliable
The authors of TFTA introduce similar algorithms, for our paper, we focus ETA which performs the best. between the
3

---

Published in Transactions on Machine Learning Research (07/2025)
and non-redundant before they are used for updating the batch-normalization parameters. To do this, they compute sample adaptive weight, ((x), for each observation before minimizing entropy:
LETA = -s(x) Σ
log f(ylz)
(2)
yEC
where S(x) is function of the entropy of the model towards the batch sample (i.e., the reliability) and the similarity to what it has seen before (i.e., non-rodundancy). Similar to the aforementioned methods, TTA minimizes entropy via sharpness-aware-nn miniiiiiiiinniioii (SAM) Foret et al. (2020). The algorithm employs high-confidence uniform sampling to create a memory bank of size NSOTTA which stores reliable and class-balanced observations. This is done by using confidence to asses if an observation should be used for adaptation. Confidence is defined as:
cxp(f(x)):
Cy(x)= = max
(3)
i j=1 exp(f(x),)
If (x)) > Co. where Co is some pre-defined confidence threshold, then x is saved into memory. Afterwards, the SAM optimizer is used to minimize entropy (equation D via two backpropagation steps. The T3A
algorithm Twasawa & Matsuo (2021) differs from the previous three as focuses on updating the prototypes Snell et al. (2017) of each class during test time:
((())
if = yk else.
S (5-11
(4)
(5)
Sp
SE
where ok represents the centroid of the prototypes of class k C, where C' is the number of classes. We define the feature extractor, 1h, all the layers of the backbone before the final dense layer. The final dense layer, Φ, is what we refer to as the classifier, it is composed of the class centroids. We denote the output of the feature extractor as 2 = (x)).
Unlike TENT, SoTTA, or ETA, T3A does not explicitly reduce entropy as it does not use a loss function, however the authors claim that entropy reduction an effect of using their algorithm. Similar to ETA, this algorithm filters less reliable samples during equation by only keeping the M lowest entropy protolypes for each class. Therefore, the algorithm stores C M prototypes.
2.2 Neural Network Calibration
Neural network calibration has been of intense interest in recent years due the critical role of confidence values, which reflect the probability assigned to predictions, in various applications. For instance, BranchyNet Teerapittayanon et al. (2017) uses neural network confidence to enable early exits for faster inference, relying on high confidence at intermediate layers. However, Zhu et al. (2022) highlights the prevalent issue of certainty calibration in deep networks, where models often display overconfidenne or underconfidence, likely due to overfitting during training. Ovadia et al.] (2019); Minderer et al. (2021) explore this concept further by measuring a model's Expected Calibration Error (ECE) and Negative Log-Liikelihood (NLL). ECE measures how closcly the confidence levels of model's predictions match the actual probability of those predictions being correct. It calculates the average absolute difference between predicted confidence and the true outcome frequencies, providing a metric for the reliability of the model's probabilistic outputs. NLL, on the other hand, captures both the model's calibration and sharpness by quantifying how well the predicted probabilities align with the observed outcomes, penalizing overconfident yet incorrect predictions more heavily. We follow their convention and use ECE and NLL as our measures for calibration error. They notice models calibrated on the validation set tend to be well calibrated on the test set, but are not properly calibrated to shifted data.
Recent work has also investigated solutions to this phenomenon. Guo (2017) discusses technique known temperature scaling while Wei et al. (222)) approaches this problem by regularizing the logit norm.
4

---

Published in Transactions on Machine Learning Research (07/2025)
Table 1: Comparison of Shannon Entropy, ECE, and NLL across the Home Office dataset domains. Results use the MobileNet backbone. Our reduction in ECE and NLL is statistically significant (p < 0.01) while maintaining competitive accuracy.
Algorithm
Shannon Entropy
ECE (lower better)
NLL (lower is better)
Art
Clipart
Product
Real World
Art
Clipart
Product
Real World
Art
Clipart
Product
Real World
No Adapt
0.950
0.951
0.704
0.689
0.502
0:316
0.183
0.169
3.166
3:330
1.777
1,638
DEC (ours
2.239
2.615
2.081
1.795
0.080
0.047
0.039
0.018
2.077
2.127
1.328
1.212
T3A
0.188
0.231
0.138
0.170
0.439
0.418
0.236
0.245
7,904
7.756
3.536
3.321
ETA
0.940
0.966
0.733
0.678
0.313
0.324
0.208
0.172
3,180
3.438
1.906
1.768
TENT SoTTA
0.918
0.895
0,653
0.636
0.299
0.308
0.187
0.172
3.120
3.2M4
1,780
1.632
0.977
0.971
0.701
0.685
0.279
0.301
0.186
0.167
2,938
3.240
1,825
1.632
More classical solutions to this problem exist as well; Zhang et al. (2021) and Müller et al. (2019) consider label smoothing to address this issue. Note that these techniques are addressed at calibrating the underlying backbone but have not been investigated with respect to TTA algorithms themselves.
2.3 Detecting Out of Distribution Data and Assessing Reliability
An increasing body of research examines how to assess whether and how closely a new observation aligns with a model's training distribution. For example, the authors of Tian et al. (001 observe that if an autoencoder was trained to reconstruct inliers, it would have a greater reconstruction error when reconstructing OOD
data. Schlegl et al. (2017) and Zenati et al. (2018) approach this issue by observing that the discriminator of
GAN learns whether not given input is an inlier. Many other works delve into this domain Bendale &
Boult ( Ming et al. (002 Park et al. (2023); Jiang et al. (2023); Wu et al. (2023) Miao et al. (2023)
Fang et al. [2022]. Regarding the TTA algorithms we compare against, the most common proxy for reliability
is entropy on the observation.
3 What is the Over-Certainty Phenomenon?
In this work, we present evidence for what we dub the over-certainly phenomenon (OCP) of contemporary TTA approaches. This phenomenon is that TTA algorithms tend to miscalibrate their underlying backbone networks by causing their predictions to be excessively certain. Modern TTA algorithms often strive to decrease test-time entropy. However, as shown in Fig. [ this entropy reduction may increase ECE and NLL because the models become overly certain on their predictions.
This phenomenon of existing algorithms causing models to become overly certain presents itself across many other datasets. A compelling example is given in Table - which agglomerates calibration errors over 15 domain shifts; in this table, we see a clear trend of entropy reduction (certainty increasing) and sub-optimal calibration. Another example is provided in Table | T3A reduces entropy by a factor of about in the art, clipart and product domains. As before, it causes ECE to worsen compared to the bascline. We do not claim that TTA algorithms should always strive to increase backbone uncertaint;; poor calibration can also be caused by under-certainty and there exist cases where reducing entropy compared baseline improves calibration. However, we find that the resulting calibration is still sub-optimal. Despite these complexities, our investigation reveals consistent pattern: the over-certainty phenomenon causes sub- sub-optimal model calibration, a significant concern for safety, robustness, and reliability.
3.1 What Causes the Over-Ceraainty Phenomenon?
We identify Iwo plausible causes of the OCP, the first issue is that modern TTA algorithms aim at minimizing backbone entropy too aggressively. In the case of TENT, ETA, and SoTTA, their loss functions, equation [ and equation [ explicitly aim at roducing a model's entropy. Regarding TENT, there is no regularization of this process. In the case of ETA, the algorithm uses a reliability score, S(z), which aims at weighing observations differently but does not regularize the distributions of the pseudo-labels. Unlike the other two,
5

---

Published in Transactions on Machine Learning Research (07 (025))
SoTTA minimizes entropy twice per iteration. The authors of T3A claim that entropy reduction is an effect of using their algorithm. In fact, they show in certain datasets T3A reduces entropy more than TENT does.
Another issue is how existing methods evaluate observation reliability, the suitability of model's prediction for use for adaptation. Previous works, ETA and T3A, tap into the power of model certainty, using it to weigh the influence of observations. ETA assesses reliability by ensuring that observations meet a certain entropy threshold; similarly, T3A uses entropy to sort the importance of class prototypes. However, prior work has shown, there are drawbacks in using entropy illustrate our point, give toy example of how using entropy can lead to misleading conclusion:
proxy for reliability in this manner Wei et al. (2022) To
Example 1. Suppose that we analyze the classifier while classifying between two classes with class centroids, co and C1. This done by taking the output of the feature extractor, tb(x) 2, and computing the dot product between the centroids and z.
g g= lz Co: 2 cc]
(6)
Consider 9e1 912 and 9s as vectors representing the inner products related to two observations, Iti and a22- and to a specific training sample, 2s. Specifically, g corresponds to the dot products between the output of the feature extractor and the class centroids. As an eromple, let's assume:
gs = (..0, ge1 [1.92, 1.00]; 9r2 = [6.10, 6.50]:
If we take the softmaz of these vectors and compute the entropy, we get Entropyy(SoftMax(]) for ges 911 and 9:2, as 0.92 bits, 0.86 bits and 0.97 bils, respectinely.
Notice that if consider the entropy of these three vectors proxy for reliability, would consider zu to be more reliable than Tı2, despite 2r2 having considerably greater inner product with the class centroids. It is highly likely that the values of occurred due to spurious feature correlations between and the class centroids. In fact, in the scenario above, F41 would be deemed to be more reliable than the genuine source domain observation Zg- Note that an analogous remark could be made on using confidence instead of entropy. Our analysis is not contrived; in Fig. [ as we increase the domain shift intensity, the observation logit norm decays and has higher variance.
Average ECE on TIN-C (EfficientNet)
Bottom 20% Quantile Logit Norm on Entire TIN-C (MobileNet)
0.20
ECE (lower better)
(Scaled Lagit. Nomn)
0.95
0.15
2.5
0.10
2.0
0.85
0.05
0.80
2
Corruption Intensity
Corruption Intensity
Cormuption Intensity
CCR T (ours)
Fixed
Figure 2: As domain shift intensity increases, the bottom quantile of the observation logit. norm de- creases. The > value (step 8) from our CCR al- gorithm represents the ratio between the norms of the observation logits, 2, and the training-sel logits, k. As z decreases, increases; this regular- izes low-logit-norm observations more aggressively (step 9). Vertical bars indicate domain-to-domain standard deviations.
Our ablation study highlights the effec- liveness of the CCR algorithu compared fixed & optimized for minimal ECE on the source-domain training set. The rightmost figure displays the com- puled values, with vertical bars indicating domain- to-domain standard deviations.
Figure 3:
Furthermore, existing approaches do not consider model's certainty on the source domain. For example, ETA's reliability paradigm rejects observations which have H 2 0.4. (C). What if we expect (i.e. there

---

Published in Transactions on Machine Learning Research (07 (075))
high entropy the training set) the model to have high entropy? By only evaluating the target domain's 's certainty without juxtaposing it against the source domain certainty, there is a lack of reference in terms of assessing the reliability of the observation. We address this issue via the to parameter explained in the next section.
4 The Dynamic Entropy Control TTA Algorithm
ameliorate the over-certainty phenomenon, we introduce Dynamic Entropy Control (DEC) (Algorithm 2 DEC refines the model's certainty levels, aligning them more closely with its actual accuracy, by selectively adjusting the temperature paramcter during the pscudo-labeling process (line 6). This is achieved without directly altering ground truth labels, instead focusing adjustments. Our approach can be implemented using single model which alternates weights. To facilitate understanding, we explain our algorithm as if it uses two distinct models: the teacher and the student. The "teacher" " model, fte, is simply our backbone before any adaptation. The "student" model, fex is the model that is adapted.
the tempering of logits through temperature
Algorithm Compute Certainty Regularizer (CCR)
Algorithm 2 Dynamic Entropy Control (DEC)
Input:
fie(X), ha, Amaee Emins Eouz,
Input: fie; fa,ho, X,K
Output: Tuee
Parameters: bmins fmazy Imazy À
1: Z = fte( (get logits}
Output: f²
Huec = (atropy)(SoftMax((2 (entropy for each
1: Time CCR(fte(X),. has Hs, Amnns lmmis Umuzz
sample}
2: Init loss
Haifi Hvec ho {compare entropy with source
3: for I; E X and Tuee do
entropy)
4: 8.i = SoftMax (x;)))
Haented = sigmoid(Hasst) Vhmas) {scale between [0,1]}
5: SE = SoftMax [(e((e) (smoothen teacher labels)
6: [ (avg(TTuae) CE(S.,,
5: Init. Toec
7: Store loss
6: for h4 Hscaled and Z do
8: end for
7:
4 = Imin + ly (lmax - (min)
9: avg(loss)
8:
7 |al (Scale the logit norm]
10: f. 0. XV
9: T. 7 {Adjust regularizer via logit norm}
11: ft = Temperature Scale(f', avg(Tuea))
10: Store Twee +
12: return ft
11: end for
12: return Tuce
Dynamic Entropy Control calibrates model predictions under distribution shift by adaptively controlling the sharpness of pseudo-labels. Instead of applying a fixed or global temperature, DEC adjusts the certainly of each pseudo-label using a scalar we call the certainty regularizer. This scalar is computed per observation, based on how uncertain the model appears on that input-taking into account both the predicted entropy and the norm of the logit vector. This mechanism allows DEC to smooth unreliable predictions more aggressively, helping the model avoid overconfident errors during test-time adaptation.
4.1 The Certainty Regularizer
The key component of DEC is the certainty regularizer, per-sample temperature scalar returned by the Compute Certainty Regularizer routine (Algorithm []. This scalar adjusts how confidently the model treats each lseudo-label. To compute T,, we first compare the model's entropy on an input 2; to its typical bchavior on the source domain. This reference entropy, hg E/Entropy (SoftMax(ffee.o. (X.)))) reflects the average certainty of the teacher model before adaptation. The difference Hxx;) ho is then passed through a sigmoid that is scaled using hmax (C), the maximum possible entropy over C classes. This yields

---

Published in Transactions on Machine Learning Research (07 (07/2025)
normalized "entropy contrast" in [o, 1], which is mapped to temperature range defined by Emin and tmaz. These bounds control the minimum and maximum smoothing applied to any sample.
To further refine this adjustment, DEC incorporates logit-norm awareness by modulating the entropy-derived temperature with the inverse norm the model's logits. Specifically, for each sample, we compute the & norm of its logit vector 2; = [tt( and scale the temperature accordingly. The scaling factor is given by "V = k//2//22 where & is the median logit norm computed over the source-domain samples. This has the effect of increasing T when the model's confidence is low in logit space, thereby injecting more smoothing into %i combining both entropy
uncertain predictions. The final certainty regularizer is then given by
contrast and logit-norm awareness to calibrate confidence per sample:
H(ri)) -ho)
T (tmin sigmoid
(tmax (min))
/mmax
T
entropy contrast (t:)
logit-norm awareness (%+
Here, (xxx the entropy of the model on Iy, bg is the average entropy on source-domain samples, 2; the logit vector, and k is the median logit norm from the source domain.
4.2 Addressing Calibration and Mitigating Pseudo-Label Noise
Our formulation allows us to interpret DEC as implicitly minimizing calibration-aware surrogate. We named T the certainty regularizer because it regulates the "sharpness" of predicted probabilities and smoothens the pseudo labels produced by the teacher. As t increases, the certainty of the prediction decreases. In other words, T should be greater for less reliable observations. This т; is used to smooth teacher predictions via temperature-scaled softmax, and the loss becomes:
LDEC
Σ VEC
8
teacher (y) log 8
student (y),
where 8 steacher (fftmaax(((e(())) and sf sttudent Softmax (fx;)))
This design has three key calibration properties:
1. Entropy contrast: The term H(x,) - ho gauges how surprising a prediction is compared to source-domain behavior.
2. Logit-norm awareness: Smaller ||z/|| values yield higher тi adding smoothing when confidence is low in logit space.
3. Sample-wise label smoothing: The softened teacher predictions act as a regularizer against overconfident or noisy pscudo-labels.
Thus, DEC behaves like an adaptive version of label smoothing, where each pscudo-label is calibrated on-the-fly based on its reliability. This implicit regularization helps discourage overconfident mispredictions and explains the empirical gains we observe in ECE and NLL.
To improve calibration after adaptation, we apply avg(Tnec)) This temperature is computed from unlabeled test. data, pre- serving the unsupervised setting. It adjusts the student's 's logits uniformly, smoothing predictions based on the average certainty regularizer. Because guides the temperature scaling component of our algorithm, we follow the example of Guo al. (2017) and mmén/mes within [1.0, 3.0]. We recommend setting both these parameters to be higher if the expected domain shift is greater. To show how DEC regularizes observations appropriately, we continue from Example [ to Example 2
final temperature scaling step:
Example 2. Given the same 911992 and 9s from Example 1 we input these into our CCR algorithm. We set by Patropyy(Soff (SoftMax tMax(gg)) [gs12, bmin 1.0 and I Imaz 2.0. Our algorithm first computes
8

---

Published in Transactions on Machine Learning Research (07/2025)
scaled entropy, sealte with respect to the source domain entropy for 9 911 and 912; as 0.49, 0.48, and 0.50, respectively.
In step 7 of CCR, this entropy is transformed into preliminary regularizer, t. Then, step 9 adjusts to by considering logit norm with respect to the source domain logit norm.
To = 1.49;
Ta: = 7.42;
Taa 1.83
Notice that, unlike purely based methods, the CCR algorithm correctly assigns greater regularization to the less reliable samples. Namcly, step 9 ensures that samples that are low-entropy due to degenerate reasons are properly regularized by considering logit norm. Furthermore, unique from existing algorithms,
our regularizer directly addresses model certainty. The impact of CCR is analyzed Fig. 3
An interpretation to how our model improves accuracy is through the works of Xie et al. (2200) and Lukasik et al. (2020). Although the former's work concerns itself in the semi-supervised learning setting, we found their observations to be relevant. That is, they introduce the noisy student, network that has been noised by dropout and stochastic-depth. They find that their noisy student can even learn to outperform the teacher which initially produced the secudo-labels For the latter work, they establish that label smoothing mitigates label noise, which is a desirable property with respect to unsupervised adaptation. Specifically, they find that label smoothing can be thought of a regularizer. This motivates us to smooth more aggressively when we suspect. that an observation might be less reliable.
5 Experiments
5.1
Experimental Setting
In order to evaluate DEC, we conduct a series of experiments using three different backbone models across four datasets. Our primary evaluation metrics will be model accuracy, NLL and ECEsins=15 on the observations, allowing us to examine both the predictive performance and the calibration quality of the models. By using varied domains and different backbone architectures, we aim demonstrate the robust and adaptability of our algorithm in handling diverse and challenging TTA scenarios. All experiments are run three times. Note that the fmin/moz hyperparameters set per dataset; do not set these per domain shift. We present accuracy and calibration for all datasets along with experiment variances either in the main paper Appendix A.5 & A.6.
We compare with TENT, T3A, SoTTA and ETA, four recent TTA algorithms which follow the test-time entropy reduction paradigm. We do a single iteration of adaptation for all algorithms unless stated otherwise. Datasot preprocessing steps and a discussion of hyperparameters for all algorithms are more detail in the Appendix A.2
5.2
Datasets
The following publicly available TTA datasets are used in our experiments; we selected these becanse they are commonly used in existing works and provide variety of domain shifts. In total, evaluate our algorithm over 26 domain shifts which include total of 282 classes. Furthermore, 15 of our domain shifts have 5 corruption levels. For some datascts, we tested using the "leave one out" (LOO) paradigm; for example, in PACS, to test generalization to pictures, we first trained our backbone networks on art, cartoon, skelch before adapting.
1. PACS Li al. 2017 has 4 domains: pictures, arl, cortoon, skelch with classes. Tested using LOO.
2. HomeOffice Venkateswara et al. (211)) has domains: art, clipart, product, real with classes. Tested using LOO.
3. Digits is combination of 3 "numbers" datascts: USPS Hull 1994)). MNIST LeCun et al. 2010 and SVHN Netzer et al. (2011). There are 10 classes. Tested using LOO by training on the source domains' training sets and adapting to target domain's 's test set.
9

---

Published in Transactions on Machine Learning Research (07 (07/2025)
4. TinyImageNet-C (TIN-C) Le & Yang 2015): has 15 domains with 200 classes. Backbones trained corruption-free (source) training set, adapted to and evaluated on corrupted (target) domains. For each target domain, there are 5 tiers of corruption.
5.3 Back Bones and Training Details
We test all but the Digits dataset on two popular classifiers, EfficientNNet100 Tan & Le (2019) and MobileNet Howard et al. (2017) pro-trained for ImageNet Deng al. (2009). We flatten the output of both networks and add a final dense layer with output shape equivalent to the number of classes.
We evaluate the Digits dataset using "SmallCNN", which is minor adaptation of the original LeNet LeCun ict al. (1998) to include batch normalization layors and max pooling. This serves to represent more compact and straightforward architectures for less complex datasets. The specific details and orderings of the layers SmalICNN are elaborated on Appendix A.7 Note that all three models use batch normalization layers as necessitated by ETA, TENT, and SoTTA.
6 Results
We present our accuracy, ECE and NLL measurements the four aforementioned datasets. To show evidence of the over-certainty phenomenon, also report prediction entropy. More comprehensive aigure/ttalls be found in the appendix. To show the impact of our CCR algorithm, which produces our certainly regularizer T, we perform an ablation experiment in Fig. 3
Table 2: Comparison Shannon entropy, ECE, and NLL averaged across the 15 domain shifts of TIN-C. We use the EfficientNet. backbone. Standard deviations across domains are shown as subscripts. This experiment highlights how excessive certainty (low Shannon entropy) correlates with sub-optimal calibration. Our reduction in ECE and NLL is statistically significant (p < 0.01) while maintaining competitive accuracy (see Tables
0 and
Algorithm
Shannon Entropy
ECE (lower better)
NLL (lower better)
Tier
Tier 2 Tier 3 Tier Tier
Tier
Tier
Tier
Tier 4
Tier
Tier
Tier
Tier 3
Tier
Tier
No Adapt
2.00
2,10mm2
2.2003
2.4b::
2.63m
0.23
0,26
0.29pmm
0.32
0.34pm:
3.06am
3.40aus
3.92am
4.57sss
5.151.0
DEC (ous)
1488
4.68aa:
4.90
5.01.
5.32
0.08
0.08
0.07pm
0.06
0.04cm
2.27am
2.44,
2.69
3.04..9
3.38.
T3A
1366n
1.41m
1.46
1,00p
1.74mm
0.33pm
0.35p
0.38
0.42gnt
0.43
3.630.
4.00pm
5.10
7:13
9.827
ETA
13Tam
1.40s
1.57a
1.7laus
1.87
0.37gax
0.40pm
0.41ga
0.43
0.44
4.45are
4.93mm
4.89as
5.63mm
6,29
TENT
1440ss
1.48a12
1.57a
1.62
1.79ha
0.27mm
0,30p
0,33mm
0.36
0,39
3.01aa2
3.30.u
3.89um
454.a
5,20
SoTTA
170mm
1.78612
188
2.0)
2.26
0,20pm
0.21a
0,23
0.25vm
0.28
2.56as
2.75.ss
3.04
3.19
3.98
6.1 DEC Reduces Calibration Error
Due to our algorithm addressing the over-certainty phenomenon, we significantly improve calibration perfor- mance. DEC achieves stato-of-the-art average ECE and NLL in all tested datasets and in nearly all individual domain shifts. We recognize that reducing entropy did improve calibration compared to baseline in some cases, but the resulting calibration was still sub-optimal. Fig. empirically validates our finding that an adaptive certainty regularizer aids in reducing ECE and NLL. Moreover, the variance of r increases as the corruption intensity increases; indicating a broador dynamic range rogularization when encountering more difficult observations.
6.2 DEC Augments Accuracy
In addition to strong calibration performance, DEC provides consistent accuracy uplifts while not necessitating any transformations on observations. By jointly exploiting backbone entropy and logit norm (see Fig. 2 able to ively assess observation reliability. After doing so, we apply greater regularization to
10

---

Published in Transactions on Machine Learning Research (07/2025)
less reliable observations. This, in turn, allows us to mitigate the potential label noise produced by the pseudo-labels.
Tables G and show that our algorithm maintains competitive accuracy with recent test-time entropy minimization approaches while addressing the over-certainty phenomenon. Moreover, unlike SoTTA and T3A, we do not store observations or training samples during the adaptation process as this could potentially cause security or privacy issues during deployment.
Table 3: Average accuracy, ECE, entropy, and NLL Digits and PACS datasets tested with LOO. Our approach achieves the best average ECE and NLL. Nomain-to-domain o values for accuracy, ECE, entropy, and NLL are reported.
Algorithm
Accuracy
ECE
Entropy
NLL
Algorithm
Accuracy
ECE
Entropy
NLL
Adapt
0.589
0.301
0.439
4.465
Adapt
0,873
0.105
0,081
1.182
DEC (ours)
0.648
0.169
DEC (ours)
0.879
0.061
0.233
0.466
T3A
0.625
1.889
T3A
0.084
0.063
ETA 0.646
0.270
0.356
3.779
ETA
0.878
0.102
0.071
1.213
TENT 0.645
0.262
0.308
3.252
TENT
0,880
0.101
0.072
1.160
SoTTA
0.640
0.252
0.474
3,222
SoTTA
0.887
0.096
0.062
1.188
maa
0.220
0.180
0.200
46.240
0.020
0.014
0.075
1.848
Table 4: Performance on the Digits dataset using the SmallCNN backbone.
Table 5: Performance on the PACS dataset using the EfficientNet backbone.
Table 6: Accuracy on TIN-C with MobileNet backbonc across different tiers of corruption. Standard deviations across the domain shifts are shown subscripts.
Algorithm
Tier 1
Tier 2
Tier 3
Tier 4
Tier 5
No Adapt
0.27po:
0.23
0.19p.07
0.15p.os
0.12g.0s
DEC (ours)
0.39
0.37
0.33
0.29
0.24g.08
T3A
0.276.01
0.24oxs
0.20g.07
0.16p.08
0.130.08
ETA
0.22goo
0.196.6
0.14
0.1
0.088
TENT
0.37
0.340.0
0.30g
0.25g.ss
0.20g.9
SOTTA
0.39
0.37
0.33po4
0.299.06
0.25p.0s
6.3 Discussion
We would like to underscore our algorithm's robustness the choice of Emin/mar- We performed our experi- ments by selecting our hyperparameters per dataset, not per domain shift. As shown in Tables 2 and we achieve statistically significant reduction in ECE despite using the same hyperparameters across 15 domain shifts. Our results in Table 1 further bolster our claims of robustness. Again, our hyperparameters are fixed across the four domain shifts but we still achieve statistically significant improvement. In the interest of re- producibility, we release our code at /attps://githbb
Our study identifies the over-certainty phenomenon of modern TTA methodologies which cause harm model calibration. To ameliorate this issue, we introduce certainty regularizer, т, that modulates model entropy and mitigates pseudo-label noise. The resulting algorithm, DEC, jointly improves model accuracy and reduces calibration error. DEC does not require batch normalization layers like SoTTA, TENT, and ETA do. This permits greater freedom when choosing a backbone. Furthermore, DEC is compatible with existing prototypical learning approaches. We speculate that there an additional impact of identifying the over-certy phenomenon: since existing work relies heavily on autropy-based techniques for assessing reliability, we surmise that improving calibration could engender improved reliability estimates. A limitation of our work is that we focus only on the classification setting; we make no claim on whether or not the over-certainty phenomenon occurs in regression scenarios.
11

---

Published in Transactions on Machine Learning Research (07 2025)
References
Martin Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, llya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda. Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning heterogeneous systems, 2015. URL https: / /www tensorflow.om org/
Software available from tensorflow.org.
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Liu, Mohammad Ghavamzadch, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243 297, 2021. ISSN 1566-2535. doi: https://doi.org/ 10. 00100611111...0 1o.///06..ii// ittp:/jjnnffnn...0000 2021.05.008
Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1563-1572, 2016.
Elliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. In International Conference on Machine Learning, pp.
PMLR, 2021.
Shai Ben David, Tyler Lu, Teresa Luu, and Dávid Pál. Impossibility theorems for domain adaptation. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 129-136. JMLR Workshop and Conference Proceedings, 2010.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Iccc, 2009.
Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? Advances in Neural Information Processing Systems, 35:37199-37211 2022.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXic:2010.01411 2020.
Tacsik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottanamurak, and Sung-Ju Lec. Sotta: Robust test-time adaplation on noisy data streams. Advances in Neural Information Processing Systems, 36, 2024.
Yunye Gong, Xiao Lin, Yi Yao, Thomas G Dictterich, Ajay Divakaran, and Mclinda Gervasio. Confidence calibration for domain generalization under covariate shift. In Proceedings of the IEEE/CVP International Conference on Computer Vision, pp. 8958 8967, 2021.
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey.
International Journal of Computer Vision, 129:1789 1819, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Ursula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (compttationaly--eenffabbb masses. In International Conference on Machine Learning, pp. 1989-1948. PMLR, 2018.
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015. URL http::/arxiw..rrg www.axwwww..rraaaa.o wwwg/aa//1503.000
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications, 2017. URL htttss://arrtet /tt/////ww..r////////////.0/00
12

---

Published in Transactions on Machine Learning Research (07 2025))
Dapeng Hu, Jian Liang, Xinchao Wang, and Chuan-Sheng Foo. caludo-aalibration: improving predictive uncertainty estimation in unsupervised domain adaptation. In Porty-first International Conference on Machine Learning, 2024.
J.J. Hull. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):550 554, 1994. doi: 1..1099344.91440.
Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems, 34:2427-244 22242, 2021.
Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han. Detecting out-of- distribution data through in-distribution class prior. In International Conference on Machine Learning, pp. 15067-15088 PMLR, 2023.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2014. URL https :
wwwrxiv.org/ab///i.. 6980
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 777):3, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278223) 2324, 1998.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann. lecun. com/erdb//////////oo///// haadbb/mmist,.. 2, 2010.
Dong-Hyun Lee et al. Psoudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 896. Atlanta, 2013.
Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy M Hospodales. Deeper, broader and artier domain generalization. In Proceedings of the IEFE international conference on computer vision, pp. 5542-5555 5550, 2017.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International conference on machine learning, pp. 6028 6039. PMLR, 2020.
Shiyu Liang, Yixuan Lú, and Rayadurgam Srikant. Enhancing the reliability of out-of-distributioo image detection in neural networks. arXiv preprint arXiv: 706. 02690, 2017.
Michal Lukasik, Srinadh Bhojanapalli, Aditya Menon, and Sanjiv Kumar. Docs label smoothing mitigate label noise? In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. Jul 2020. URL https::/proceediing mlr wttppp/////////ksskkk0.... html
PMI.R, 13-18
Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Best sources forward: domain generalization through source-specific nets. In 2018 25th IEEE international conference on image processing (ICIP), pp. 1353 1357. IEEE, 2018.
Wenjun Miao, Guansong Pang. Tianqi Li, Xiao Bai, and Jin Zheng. Out-of-distributtion detection in long-tailed recognition with calibrated outlier class learning. arXio preprint or Xiii:2312210068 2023.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in Neural Information Processing Systems, 34:15682-15694, 2021.
Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of-dist detection with vision-language representations. Advances in Neural Information Processing Systems, 35: 35087-35102, 2022.
13

---

Published in Transactions on Machine Learning Research (07/2025)
Rafacl Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? CoRR, abs/1906.026222 02629, 2019. URL.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. Neurips Workshop on Deep Learning, 2011.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International conference on machine learning, PMLR, 2022.
pp.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan,, and Jasper Snock. Can you trust your model's uncertainty? evaluating prodictive uncertainly under dataset shift, 2019. URL http:://a/xxxi..oot
http://axwww.oggbbb////asa atgg/bss/900..... 02530
Anusri Pampari and Stefano Ermon. Unsupervised calibration under covariate shift. arXiv preprint arXiv:2006.1644 16405, 2020.
Jaewoo Park, Jacky Chen Long Chai, Jaeho Yoon, and Andrew Beng Jin Teoh. Understanding the feature norm for out-of-distribution delection. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 1557-1567, 2023.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
Ori Press, Ravid Shwartz-Ziv, Yann LeCun, and Matthias Bethge. The entropy enigma: Success and failure of entropy minimization. arxie prepriul ww.Xv:2240.008 050112 2024.
Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, and Andrew Ng. Self-tanght learning: transfer learning from unlabeled data. In Proceedings of the 24th international conference Machine learning, pp- 759 766, 2007.
Thomas Schlegl, Philipp Sceböck, Sebastian M. Waldstein, Ursula Schmidt-Erfurtha and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. CoRR, abs/1703.055211 2017. URL.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.
Samuel Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew G Wilson. Does knowledge distillation really work? Advances in Neural Information Processing Systems, 34:6906-660 6919,, 2021.
Mingxing Tan and Quoe V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR, abs/1905. 11946, 2019. URL http:p///arri..orrab//10905 11946.
Hui Tang, Ke Chen, and Kui Jia. Unsupervised domain adaplation via structurally regularized deep clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8725 8735, 2020.
Surat Teerapitlayanon, Bradley McDanel, and H. T. Kung. Branchynet: Fast inference via early exiting from 01686
deep neural networks. CoRR, abs/ Abs//100.000 1709.01666. 2017. URL
Kai Tian, Shuigeng Zhou, Jianping Fan, and Jihong Guan. Learning competitive and discriminative reconstructions for anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 5167 5174, 2019.
Tijmen Ticleman, Geoffrey Hinton, et al. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent. magnitude. COURSERA: Neural nelworks for machine learning, 4(2):26 31, 2012.
14

---

Published in Transactions on Machine Learning Research (07/2025)
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEF conference on computer vision and pottern recognition, pp. 5018 5027, 2017.
Yoav Wald, Amir Feder, Daniel Greenfeld, and Tri Shalit. On calibration and out-of-domain generalization. Advances in neural information processing systems, 34:2215-2227, 2021.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint vxXi:2206.10722 2020a.
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip Yu. Generalizing to unseen domains: survey on domain generalization. IEEE Transaclions on Knowledge and Data Engineering, 2022.
Ximei Wang, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable calibration with lower bias and variance in domain adaptation. Advances in Neural Information Processing Systems, 33:19212 199-9222-11228,,
2020b.
Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. In International Conference Machine Leurning, pp. 23631-22222000 23644. PMLR, 2022.
Xinheng Wu, Jie Lu, Zhen Fang, and Guangquan Zhang, Meta ood learning for continuously adaptive ood detection. In Proceedings of the IEEE/CVP International Conference Computer Vision, pp. 19353-1930) 2023.
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoe Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 10687-10698, 2020.
Yurong You, Cheng Perng Phoo, Katie Luo, Travis Zhang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger. Unsupervised adaptation from repeated traversals for autonomous driving. Advances in Neural Information Processing Systems, 35:27716-25 27729, 2022.
Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, and Vijay Ramaseshan Chandrasekhar. Efficient gan-based anomaly detection, 2018. URL attpp://arxils http:///aaxxioooraab/////// http//2b//180/..../ 1111//0222002222222
Chang-Ba Zhang, Peng- Tao Jiang, Qibin Hou, Yunchao Wei, Qi Han, Zhen Li, and Ming-Ming Cheng. Delving deep into label smoothing. IEEE Transactions on Image Processing, 30:5984 5996, 2021. doi: http://tpp.221.888000000
URL https://doi.orgg:
11t9%/2tip.200000 208..399942
Weicheng Zhu, Matan Leibovich, Sheng Liu, Sreyas Mohan, Aakash Kaku, Boyang Yu, Laure Zanna, Narges Razavian, and Carlos Fornandez-Granda. Deep probability estimation, 2022. URL https://openreviie
net/forum?id-hone v7Hvvz
15

---

Published in Transactions on Machine Learning Research (07 (025)
A Appendix
A.1 Experimental Setup Details
We used TensorFlow 2.9 Abadi et al. 2015 with Nvidia CUDNN version 11.3 on an RTX 3080 16GB laptop GPU with 32GB of system memory. All experiments are run three times using random_seed = 0,1,2 respectively.
1. PACS Li et al. (2017) has 4 domains: pictures, art, cartoon, sketch with 7 classes. All images are resized to (227, 227,3 and scaled between [0, 255]. Tested using LOO. For both MobileNet and NeficientNet: We set tmin and Emaz parameters to 1.00 and 3.0 respectively.
2. HomeOffice Venkateswara et al. (017) has 4 domains: art, clipart, product, real with 65 classes. All images are resized to (128, 128, 3) and scaled between [0, 255]. Tested using LOO. MobileNet: We set fii and Imaz parameters to 1.20 and 2.75 respectively. EfficientNet: We set biin and Imar parameters to 1.00 and 1.75 respectively.
3. Digits is combination of 3 "numbers" datasots: USPS Hull 1994.. MNIST LeCun et al. 2010) and SVHN Netzer et al. (2011). The images are resized to (32, 32, 1) and scaled between [0, 255]. There are 10 classes. Tested using LOO by training on the source domains' training sets and adapting to target domain's test set. SmalICNN: We set bmé and fmar to 1.20 and 2.75, respectively.
4. TinylmageNet-C (TIN-C) Le & Yang (015)) , has 15 domains with 200 classes, All images are resized to (256,256,3)) 3) and scaled between [0, 255]. Backbones are trained on corruption-free (source) training set, adapted to and evaluated on corrupted (target) domains. For each target domain, there are 5 tiers of corruption. MobileNet: We set Emá and fma parameters to 1.50 and 3.00 respectively. Neficienttee: We set Emin and tmaz parameters to 1.00 and 2.50 respectively.
A.2 Hyperparameters
We do most initial training on the source domain using RMS Prop(1r = 2e 44 Tieleman et al. (2012) to minimize coos-eenroppy loss for epochs {15, 15, 25) for each enumerated dataset, respectively. SmallCNN is compiled and initially trained with the Adam optimizer Kingma & Ba (2014) in lieu of RMSProp. We estimate roughly 1,000 hours of GPU usage at 130 watts of power to conduct that MobileNet expects inputs to be prepossessed in unique manner. We use Tensorflow's off-the-shelf pre-processing layer for MobileNet at the input. For the tests of statistical significance, we compared DEC with the second best algorithm for each experiment.
experiments. Note
Note that the following are parameters for what we compare against. Further clarification on their meaning can be found in their respective works. For ETA, we set 0 0.4 In (C), as this was 0.4, (0.6, 0.1, 0.4, 0.125) for each enumerated dataset, respectively. These
their recommended value, and values were empirically chosen to help their performance. For T3A, we set the number of supports to relain, M = 0o, as this provides the lowest calibration error. For SoTTA, we set = 0.05, C, = {0.33, 0.33, 0.33, 0.66] for each dataset respectively to help their performance, and NSATTA = 64 as per their recommendations. We use the authors' recommended batch sizes for all techniques.
We selected hyper parameters either by the recommendation of the respective authors' or order improve performance with respect to calibration error. For example, for ETA we experimented with various values for in order to lower their ECE much as possible per dataset (not per domain shift). A similar process used for selecting Co to aid SOTTA
For our own hyper parameters, we started with twin = 1.0 and tmax = 3.0, then tuned them as we did the other works. We want to emphasize the robustness of our algorithm to the seloction of tmin/mea- Our hyperparameters are chosen for each dataset, not for individual domain shifts. As shown in Tables 2 and 7 we achieve statistically significant reduction in ECE. and NL. despite using consistent.
16

---

Published in Transactions on Machine Learning Research (07 (07/2025)
hyperparameters across 15 domain shifts. Our results in Table [ further support claims of robustness, showing statistically significant improvement even with fixed hyperparameters across the four domain shifts. Another note is that although step 6 of DEC (Algorithm 2 shows the teacher model inferencing, we can actually just store and reuse the logits computed in line 1 of the CCR Algorithm pass compulation. We use a batch size of 50 for our DEC for all experiments.
This saves us one forward
A.3 Further Discussion of Related Work
As stated previously, the focus of our work is on TTA algorithms. However, we would like to discuss some studies on OOD calibration to provide supplementary background. The authors Wang et al. (2020b) and Gong et al. (2021) remark on poor calibration in the OOD sctting, however, according their algorithms, they require the
observations during training. That is, their algorithm not for the deployed setting like ours is. Another work, Wald et al. (2021p, discusses proxies for determining if model would be calibrated during deployment, before deployment. Finally, there exist works such as Hu et al. (2244) which are post-hoc calibration techniques (not TTA algorithms). We recognize the relevance of these works but emphasize that they address different facets or settings.
Recent work by Press et al. (202)) offers complementarr view on the failure modes of entropy minimization under domain shift. Their study shows that while early adaptation clusters test embeddings near training data and boosts accuracy, continued optimization pushes embeddings away leading accuracy collapse. This biphasic behavior helps explain why TTA methods often degrade in calibration time. While their focus is on representation drift and unsupervised accuracy estimation, our work highlights a parallel failure mode: over-adaptation also induces epistemic miscalibration.
A.4 Memory Tradeoffs
The memory overhead of T3A, PI34, is function of their hyperparameter M and the size of the final classifier. For SoTTA, the memory overhead is the same as TENT/ETA plus the NSATTA observations it stores. More formally, by defining that ow as the number parameters the feature extractor and classifier, respectively:
Ртал = M twu
(7)
PDEC 2 (Nw + Φw)
(8)
PETA PTENT NumBNParams
(9)
PSOTTA NumBNParams NSOTTA
(10)
Note that the frozen copy of the backbone (teacher model) used in DEC does not need to be in GPU VRAM at the same time as the backbone being adapted.
17

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A.5 Additional Experiments
Table Comparison of Shannon entropy, ECE, and NLL averaged across the 15 domain shifts of TIN-C. We the MobileNet backbone. Standard deviations across domains shown subscripts. This experiment. highlights how excessive certainty (low Shannon entropy) correlates with sub-optimal calibration. Our reduction in ECE and NLL is statistically significant (p < 0.01) while maintaining competitive accuracy.
Algorithm
Shannon Entropy
(lower better)
(lower
Tier
Tier
Tier
Tier
Tier
Tier
Tier
No Adapt
1.46.1
1,53as
1.5eas
166ocs
1.00g.21
0.40oas
0.446as
0.47aos
11500
11.53ue
5,98
6.50use
7,30
8.42g.44
9.50g.
DEC (ours)
21
3.55azo
3,80uu
3.96.4
4.04a.3
0.05ma:
0.05eaa
0.06:
0.07cm
0.10cmc
2.08.11
2.83,17
3.05m
3.3Tc.1t
3.78ams
188ast
1876.3
1889.
0422ua
0.44aaz
0.46.a
0.48ux
8.50um
6,80
7.15
9:37aa
18.275:3
19.00ss
RTA
1.1020
1.22an0
1.36as5
1.39cca
1.34,3
0.54aas
0.562as
0.50a
0.02axe
8.00a
8.03use
9.832
10.682.22
12.02.0
TENT
1.10ms
1.1um
1.2kuw
122a.a
1.34.z
0.38uaa
0.40laus
0:43,u
0.47am
0.50um
4.9%ua
5.47uxs
6.25,
7.36
8.5asa
1.10m
14
0.30ca:
0.45mre
6.10vm
Table 8: Accuracy on TIN-C with EfficientNet backbone across different tiers of corruption for various algorithms. Standard deviations shown as subscripts.
Algorithm
Tier
Tier 2
Tier 3
Tier
Tier 5
No Adapt
0.41o.04
0.37p.0e
p.31
0.240.00
0.196.10
DEC (ours)
0.49u.u2
0.46..01
0.41ums
0.35u.1
0.29h.11
T3A
0.420.04
0.38p.co
0.32p.os
0.26o.10
0.21o.10
ETA
0.33
0. 1.29
0.266.07
0.22o
0. 1.199.0e
TENT
0.47a.as
0. 422..s
0.37n.ss
0.31o.10
0. 0.250.11
SOTTA
0.49n.u2
0.46a.00
0.42o.06
0.36
0.316.10
Table 9: Average accuracy, ECE, entropy, and NLL on the Home Office dataset using the Efficient backbone. Domain-to-domain o values for accuracy, ECE, entropy, and NLL are reported.
Algorithm
Accuracy
ECE
Entropy
No Adapt
0.665
0.198
0.642
2.184
DEC
0.609
0.069
1.832
T3A
0.686
0.267
0.186
4.321
ETA
0.665
0.218
0.422
2.745
TENT
0.684
0.211
0.460
2.396
SoTTA
0.194
0.030
18

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Table 10: Average accuracy, ECE, entropy, and NLL on the Home Office dataset using the MobileNet
backbone. nomain-to-domail max values for accuracy, ECE, entropy, and NLL are reported.
Algorithm
Accuracy
ECE
Entropy
NLL
No Adapt
0.567
0.242
0.824
2.184
DEC
0.575
0.046
2.183
1.686
T3A
0.617
0.335
0.182
4.321
ETA
0.553
0,254
0.807
2.745
TENT
SoTTA
0.574
0.233
0.833
2.409
max
0.013
0.011
0.116
6.467
Table 11: Average accuracy, ECE, entropy, and NLL on the PACS dataset using the MobileNet backbone. Domain-tor Domain-to-domail max values for accuracy, ECE, entropy, and NLL are reported.
Algorithm
Accuracy
ECE
Entropy
NLL
No Adapt
0.841
0.111
0.177
1.182
DEC (ours)
T3A
0.857
0.116
0.086
0.866
ETA
0.842
0.101
0.192
1.213
TENT
0.848
0. 0.108
0.162
1.160
SOTTA
0.852
0.102
0.180
1.188
0.034
0.015
0.099
1.016
A.6 Run-to-Rum Variances across TTA algorithms
For the Digits dataset: a = [4.39 10 10-3 5.51 x 10-3, and entropy, respectively across all trials using SmalICNN.
3.85 3.85
10 1.000 x 10-3] for accuracy, ECE,
For Home Office across both backbones: o |5.00 x 10-?, 1.12 10-5 1.66 x 10-5, 9,299
7.29 x 10 10-3)
for accuracy, ECE, and entropy, respectively.
For the PACS dataset across both backbones: 0 1 10 for accuracy, ECE, and entropy, respectively across all trials.
x 10 1.02
2.09
2.00 10
For the TIN-C dataset across both backbones and across all domain shifts: - |1.39 10 1.12x 10 10-7,2.46 x 10-3 2.40 10-1] for accuracy, ECE, and entropy, respectively across all trials.
19

---

Published in Transactions on Machine Learning Research (07 (07/202) (025)
A.7 SmallCNN
ingtt
[Coone, 32. 32, 1)]
Tepert
ouryne:
[Noone, 32, 32,
Iign::
(Nome, 32, M2,
ontput
(Noue, 32, 52, 32)
aqpal
Nome: 32, 32, 32)
Ratehikoommilatioon
oulpuat
Nomee 32, 32, 12)
tipar
(Ncne, M,, M2, 10)
Actrvation
coppa
(Ncue, 32, 22)
ligun:
(Nouc, 32, 82)
CamedD
ontput:
(Nouc, 32, 64)
aiput
[Ncue 64)
BrtttaN-oomaaaation
cuppat
32, 64)
kipar
Activation
colper
(Newe
(Mone.
MastPollaggD
oulput
(Moue,
Comp2D
64) (Mone,
auput
(Noue 16, 64)
Betchhoomadaatio
oulpua
CNuue 16, 16, 64)
xipat
DNeut 10, 16,04)
curpet
Dicue, C4)
inpt:
(None, 18, 16, 64)
MasPooolnge
uulput
(None, 8. 8, 64)
iggi::
(Noaa,, 6
Flatten
onejut:
(Noue, 4090)
Ingut:
(None: anss)
Dause
output
(Koon, 6.))
input
(None,
DatehNomaaizaton
oulput
64)
Ingei:
(Noue,
Activation
outpet
(Mous,
Flatten
corpont
6.1)
Dennce
(Iooow, 10)
Figure 4: SmalICNN's architecture has 319,498 parameters.
20


Reviews:
[{"review_id": "nTzQZjE88C", "reviewer": "Reviewer_iKPA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**\n\n- Investigates the calibration issue in entropy-minimizing test-time adaptation algorithms\n\n- Offers mechanistic analysis of model certainty behavior during adaptation\n\n- Provides comprehensive empirical evaluation across multiple datasets, domain shifts, and model architectures\n\n**Weakness**\n\nMy main concern lies in two points:\n\n- The experimental setting should be validated on larger dataset benchmarks, as the current evaluation is relatively small. TTA is a lightweight task, and we tend to adapt only very small numbers of trainable parameters, as well as run on each example just one or a few times.\n\n- Authors should explore additional applications of their method in TTA settings where calibration offers greater benefits to highlight their method's importance - for example, applying their approach to improve sample selection in TTA, which currently relies on entropy-based reliability measures", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper identifies the \"over-certainty phenomenon\" in test-time adaptation algorithms, where entropy minimization leads to miscalibrated predictions under domain shift. It proposes Dynamic Entropy Control (DEC), which improves model calibration by adaptively adjusting prediction certainty based on entropy and logit norm analysis. Extensive experiments across domain shifts demonstrate state-of-the-art calibration performance (measured by ECE and NLL) while maintaining competitive accuracy. The work provides both mechanistic insight into why current TTA methods struggle with calibration and a practical solution to address this limitation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the Weakness.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5wTrybriNH", "reviewer": "Reviewer_F8Wi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. This paper studies the calibration probelm in OOD setups, which is critical in real-world scenarios. In addition, the results show an impressive improvement. \n2. There are clear motivations for this probelm, and the solution is also valid. \n\nWeaknesses\n1. Though the test-time adaptation is the main topic, the conventional calibration approaches, such as temperature scaling and in-processing algorithms, can be baselines that are not thoroughly studied. \n2. In the algorithms, there are some terms not clarified, which raises difficulties in understanding the results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers the calibration probelm in the OOD task, which can amplify the over-confidence/under-confidence probelms. Usually, the entropy is used to assess the uncertainty of pseudo-labeling. However, this can have limitations in the lack of comparison with source entropy and a more detailed strategy. The paper proposed a new algorithm addressing the over-confidence/under-confidence probelm in test-time adaptation algorithms. The results look promising, and DEC's algorithm can resolve the over-confidence/under-confidence problem in many datasets and tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Page 7\n\n- $SoftMax_{T=1}$ and $SoftMax_{T=\\tau_i}$ are not defined theruoghly. \n- $Temperature-scaling$ $(f^{'}_s,  {avg} (T_vec) )$ is not defined theruoghly. \n\nThese two terminologies are typical in the classification task. However, more clarified definitions are required for a more understandable paper.\n\nIn Table 6: Accuracy on TIN-C, the accuracy is overall lower; maybe the task can be difficult. In this case, the advantages of DEC in calibration are not well-presented. It is better to present some clues about these advantages.", "broader_impact_concerns": "There is no critical issue on Broader Impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8F37hOaz5W", "reviewer": "Reviewer_56nD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths: \n1. Thorough Experiments: Extensive empirical validation across diverse datasets and backbones; statistical significance is reported.\n2. Robustness: Demonstrates that DEC performs consistently across different corruption intensities and domains without per-domain tuning.\n\n\n## Weaknesses: \n1. Over-cartainty problem, is often called overconfidence problem has been studied in this field.\n2. Lack of novelty, the identification of over-certainty problem is discussed by some works, for example [1]\n3. Writing needs to be improved, for example, the abstract should provide enough information for user to understand the work. However, how the method works is not mentioned in the abstract\n4. Most analysis are based on heuristic, \n5. Is there any instruction for the hyper parameter t_max, t_min?\n6. What if the BN is replaced by other things such as layer norm or group norm.\n \n\n\n\n\n[1] Pseudo-Calibration: Improving Predictive Uncertainty Estimation in Unsupervised Domain Adaptation", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper identifies and studies the over-certainty phenomenon (OCP) in modern test-time adaptation (TTA) algorithms, where entropy-minimization-based methods unintentionally worsen model calibration under domain shift. While previous work focuses primarily on accuracy gains during TTA, this study is the first to systematically highlight how these methods can drive models to become overly confident, exacerbating calibration errors. To address OCP, the authors propose Dynamic Entropy Control (DEC) — a novel TTA method that dynamically adjusts the confidence of pseudo-labels during adaptation based on the relative entropy and logit norm of observations compared to the source domain. DEC introduces a certainty regularizer that adapts temperature scaling individually per sample. Comprehensive experiments across 26 domain shifts and several datasets (e.g., PACS, HomeOffice, Digits, TinyImageNet-C) show that DEC consistently reduces expected calibration error (ECE) and negative log-likelihood (NLL) while maintaining or improving accuracy. The method is also shown to be robust across datasets with fixed hyperparameters, without the need to store test-time data.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add theoretical justification or more intuition for the τ scaling \n2. Improve the abstract as mentioned above\n3. Report runtime and memory overhead compared to TENT, SoTTA.", "broader_impact_concerns": "Impactful Practical Implications: Highlights why calibrated uncertainty is critical in real-world safety-critical applications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)