Label Distribution Shift-Aware Prediction Refinement for Test-Time Adaptation
Authors: Minguk Jang
Hye Won Chung
Venue: Accepted by TMLR ()

Abstract: Test-time adaptation (TTA) is an effective approach to mitigate performance degradation of trained models when encountering input distribution shifts at test time. However, existing TTA methods often suffer significant performance drops when facing additional class distribution shifts. We first analyze TTA methods under label distribution shifts and identify the presence of class-wise confusion patterns commonly observed across different covariate shifts. Based on this observation, we introduce label Distribution shift-Aware prediction Refinement for Test-time adaptation (DART), a novel TTA method that refines the predictions by focusing on class-wise confusion patterns. DART trains a prediction refinement module during an intermediate time by exposing it to several batches with diverse class distributions using the training dataset. This module is then used during test time to detect and correct class distribution shifts, significantly improving pseudo-label accuracy for test data. Our method exhibits 5-18% gains in accuracy under label distribution shifts on CIFAR-10C, without any performance degradation when there is no label distribution shift. Extensive experiments on CIFAR, PACS, OfficeHome, and ImageNet benchmarks demonstrate DART's ability to correct inaccurate predictions caused by test-time distribution shifts. This improvement leads to enhanced performance in existing TTA methods, making DART a valuable plug-in tool.

Published in Transactions on Machine Learning Research (02/2025)
Label Distribution Shift-Aware Prediction Refinement for
Adaptation
Minguk Jang
mgjongGkaist.oo.ktt
School of Electrical Engineering, KAIST
Hye Won Chung
hucckung@kkisst.oom
School of Electrical Engizeering, KAIST
Reviewed on OpenReview: https: // openreview. net/ pormforum? C c7AHHdY
Abstract
Test-time adaptation (TTA) an effective approach mitigate performance degradation of trained models when encountering input distribution shifts at test time. However, existing TTA methods often suffer significant performance drops when facing additional class distri- bution shifts. We first analyze TTA mods under label distribution shifts and identify the presence of class-wisc confusion patterns commonly observed across different covariate shifts. Based on this observation, we introduce label Distribulion shift- Aware prediction Resinemenl for Test-time adaptation (DART), a novel TTA method that refines the predictions by focus- ing on class-wise confusion patterns. DART trains a prodiction refinement module during an intermediate time by exposing it to several batches with diverse class distributions using the training dataset. This module is then used during test time to detect and correct class distribution shifts, significantly improving pseudo-label accuracy for test data. Our method exhibits 5- 18% gains in accuracy under label distribution shifts on CIFAR-10C, without any performance degradation when there is no label distribution shift. Extensive experiments on CIFAR, PACS, OfficeHome, and ImageNet benchmarks demonstrate DART's ability to correct inaccurate predictions caused by test-time distribution shifts. This improvement leads to enhanced performance in existing TTA methods, making DART valuable plug-in tool.
1
Introduction
Deep learning has achieved remarkable success across various tasks, including image classification (Krizhevsky
ct al.
2012 Radford et al. 2021 SSmonyya & Zisscrman 2014) and natural language processing (Devlin et al.
2018
Vaswani al.) 2017). However, these models often suffer from significant performance degradation
when there is a substantial shift between the training and test. data distributions (Mendonca et al. 2020
Saenko et al. 2010 Taori et al. 2020). Test-time adaptation (TTA) methods [Boudiaf et al. 2022 Goyal
ct al. 2022 Jang et al. 2022 Wang et al.) 2020 Zhao et al.] 2022) have emerged as a prominent solution to
mitigate performance degradation due distribution shifts. TTA methods allow trained models to adapt
the test domains using only unlabeled test data. In particular, an approach known as BNAdapt (Nado et al.
2020: Schneider al. 2220),, which substitutes the batch statisties in the batch normalization (BN) layers of trained classifier with those from the test batch, has proven to be effective in adapting to input distribution
shifts in scenarios such as image corruption. Consoquently, numerous TTA methods (Lee. 2013
Wang et al.
2020 Zhao et al. 2022 Zhou et al. 2023 are built upon the ANAdapt strategy.
However, recent studies (Gong et al. 2022 Niu et al. 2023 Zhou et al. 2023) have shown that the effectiveness
of the BNAdapt strategy diminishes or even becomes harmful when both the class distribution and the input distribution in the test domain shift at test time. This reduction in effectiveness is due to BNAdapt's reliance batch-level data statistics, which influenced not only by class-conditional input distributions but.

---

Published in Transactions on Machine Learning Research (02/2023)
COALE SEANLE
CHALEC
Geste
82
Reatl
Spoll
STARL
10-10
R-50
4
2e-201
Figure 1: Confusion patterns of BN-adapted classifier due to test-time label distribution shifts. We present class-wise confusion matrices of BN-adapled classifiers, initially trained on class-balanced CIFAR- 10 and then tested on CIFAR-10C with two long-tailed distributions (first column). The second column shows confusion patterns on the C1FAR-10 test datasct with only label shifts, while the third to fifth columns display patterns on CIFAR-10C under three types of corruptions combined with label shifts. Class pairs where the confusion rate exceeds 11% are highlighted in red. There is significant accuracy degradation in head classes (e.g,, class 0 in the 1st row and class 9 in the 2nd row). Additionally, similar class-wise confusion patterns are observed across different types of corruption for each label distribution shift (cach row). Confusion matrices for other 15 types of corruption and class imbalance ratios of p = 1, 10, 100 are also reported in Figure
also by the configuration of data within the test batches. For instance, when a test batch predominantly contains samples from a few head classes, the batch statistics become biased towards these classes. To address these limitations, some recent methods have been designed to lessen the dependency of test-time adaptation strategies on batch-level statistics and to tackle class distribution shifts with additional adjustments. For instance, NOTE (Gong et al.; 2022) employs instance-aware batch normalization, which diverges from traditional batch normalization, and uses a prediction- balanced memory to simulate class-balancee test data stream. SAR (Niu et al. 2023) implements batch-agnostic normalization layers, such as group or layer norm, complemented by techniques designed to drive the model toward a flat minimum, enhancing robustness against noisy test. samples. However, these methods still rely on pseudo labels of test samples, and their effectiveness is fundamentally limited by the accuracy of these pseudo labels, which particularly deteriorates under severe label distribution shifts.
In this work, we propose a more direct, simple yet effective method that can correct the inaccurate predictions generated by the BNAdapt strategy under label distribution shifts. Our method can be integrated with a
Niu et al., 2023 Schneider et al. 2020 Wang et al. 2020 Zhao et al. 2022 Zhou et al. 2023 significantly enhancing their accuracies by 5-18 percentage points under label distribution shifts, while having minimal impact when label distributions remain unchanged. We achieve this by first identifying the existence of class-wise confusion patterns, i.e., specific misclassification trends between classes that the BNAdapt classifier exhibits under label distribution shifts, regardless of different input corruption patterns (Fig, [ In particular, we show that these confusion patterns depend not only on the inter-class relationships but also on the magnitude and direction of the label distribution shifts from a class-balanced training dataset. Building on this insight, it might seem natural to correct the BN-adapted classifier's 's inaccurate predictions by applying a simple affine transformation that reverses the effects of class-wiss wise confusion as in learning with label noise (Natarajan et al.) 2013 Patrini et al. 2017 Zhu al.] 2021). However, a significant challenge arises because these class-wis confusion patterns are difficult. to deduce using only the unlabeled test data, which has unknown label distributions. Additionally,
variety of existing TTA methods (Boudiaf et al. 2022 Gong et al. 2022 Lee 2013
2

---

Published in Transactions on Machine Learning Research (02/2025)
D Drain
Prediction refinement module
Logit 1
fa
DB R
Y/
g
Shared
We E & by BANK
LE
dg E R
Gradient
fø
PB. d
High: mild shift Low: severe shift
S
Refined prediction: @: softmax(), W// + bg)
Diverne lobal distribttion via simplig
Figure 2: Intermediate time training of DART. At intermediate time, the period between the training and test times, DART trains a prediction refinement module Ie to correct the inaccurate prediction caused by the class distribution shifts. (left) By sampling the training data from Dirichlet distributions, we generate batches with diverse class distributions. (right) The prediction refinement module Io takes the averaged pseudo label distribution Pg and prediction deviation dg for each batch B, and outputs a square matrix WB and a vector by of size K (class numbers). Using labels of the training dala, we optimize go to minimize the cross-entropy loss between labels y and the refined prediction q softmax(f(())Ws + bB) for samples (x, y) E B for the BN-adapted classifier fø-
these patterns can vary over time with online label distribution shifts, further complicating the adaptation process.
To lackle these challenges, we propose a novel method, named label Distribulion shist-Aware prediction Refinement for Test-time adaptation (DART), which detects test-time label distribution shifts and corrects the inaccurate prodictions of BN-adapted classifier through an effective affine transformation depending the label distribution shifts. Our key insight is that the model can learn how to adjust inaccurate predictions caused by label distribution shifts by experiencing several batches with diverse class distributions using the labeled training dataset before the start of test time. DART trains a prediction refinement module during an intermediate time, positioned between the end of training and the start testing, by exposing multiple batches of labeled training data with diverse class distribbutions, sampled from the Dirichlet ristribution (Figure 2 The module uses two inputs to detect label distribution shifts: the averaged pseudo-label distribution of each batch generated by the BN-adapted classifier and a new metric called prediction deviation, which measures average deviations of each sample's soft pseudo-label from uniformity to gauge the confidence of predictions made by the BN-adapted classifier. The module is trained to generate two outputs, square matrix and a vector of class dimensions, which together transform the logit vector for prediction refinement. Since this module requires only soft pseudo-labels for lest samples, it can be readily employed during Lest. time using the pre-trained BN-adapted model. This approach enables DART to dynamically adapt to label distribution shifts at test time, enhancing the accuracy of predictions.
We evaluate the effectiveness of DART across wide range of TTA benchmarks, including CIFAR-10/100C, ImageNet-C,, CIFAR-10.1, PACS, and OfficeHome. Our results consistently demonstrate that DART signifi- cantly improves prediction accuracy across these benchmarks, particularly in scenarios involving tost-time distribution shifts in both input and label distributions. This enhancement also boosts the performance of existing TTA methods, establishing DART as a valuable plug-in tool (Table [] Specifically, DART achieves notable improvements in test accuracy, enhancing the ANAdapt method by 5.7% and 18.1% on CIFAR-10C-LT under class imbalance ratios of p 10 and 100, respectively. Additionally, our ablation studies highlight the critical role of the prediction refinement module's design, particularly its inputs and outputs, in enhancing DART's effectiveness.
2 Label Distribution Shifts on TTA
TTA methods are extensively investigated mitigate input data distribution shifts, yet the impact of label distribution shifts on these methods is less explored. In this section, we theoretically and experimentally
3

---

Published in Transactions on Machine Learning Research (02/2025)
analyze the impact of label distribution shifts on BNAdapt (Nado et al. 2020
Schneider et al. 2020):
ional strategy for many TTA methods [Lee] 2013 Wang et al. 2020 Zhao et al. 2022: Zhou et al.
2023). We also introduce a new metric designed to detect label distribution shifts at test time using only unlabeled test samples.
Impact of label distribution shifts on TTA We begin with a toy example to explore the impact of test-time distribution shifts on a classifier trained for a four-class Gaussian mixture distribution. We assume that the classifier is adapted at test time by mean centering function; modeling the effect of batch normalization in the BNAdapt. Consider the imput distribution for class i as (N; 1212, where Ai (d, Bd), 222 (-d, Bd), #3= = (dd, and μl4 = (-d, -Bd) with d E E and | >> 1. Assume uniform priors Ptr = [1/4.1/4, 1/4, 1/4] for training. At test time, we consider shifts in both imput and label distributions. Consider the covariate shift. by A E TR2 as studied in prior works (Stojanov et al. 2021 Yi et al. 2023). This changes the input distribution for each class to Niu + A, I2).. Additionally, let the class distribution shift to Pt. (p. 1/4, 1/4,1/2 p. for some p E [1/4.1/2) Let h(-) denote a Bayes classifier for the training distribution, and Norm(-) represent a mean centering function, mimicking the batch normalization at test time. When p 1/4 (i.e., Per Pte), the mean centering effectivcly mitigates the test-time input distribution shift, restoring the original performance of l(-). However, when p > 1/4 (i.e., Per + Pta), the ad-ated classifier, modeled by h(-) == h(Norm(-)), begins to exhibit performance degradation with a distinct class-wise confusion pattern. This pattern reflects both the spatial distances between class mcans and the severity of the label distribution shifts:
(#1) The misclassification probability from the major class (class 1) to a minor class is higher than the reverse: Plz-N((u+dd."%1)) (xxx) = > Pr.-W(p:+S.o (hx)) = 1]. #1
(#2) The misclassification patterns are influenced by the spatial relationships between classes, e.g., the probability of misclassiTying from class 1 to other classes is higher for those that are spatially closer: Pio-N(nn+s [hxx) 2 Pla-N(yn+t.. 12) [k(x) = 3] > PronN((n = 4].
(#3) As the class distribution imbalance p > 1/4 increases, the rate at which misclassification towards spa- á tially closer classes increases is greater than towards more distant classes: öp Plo-Nont4 [h(x)
2] >
a Pra-N(un+S. [h(x) = 3].
Up
More additional explanations for these confusion patterns including the symbol table, illustrations, and detailed proofs are described in Appendix D Through experiments with real datasets, reaffirm the impact of label distribution shifts on a BN-adapted classifier. We consider BN-adapted classifier originally trained on a class-balanced CIFAR-10 dataset (Krizhevsky & Hinton 2009) and later tested using CIFAR- 10-LT and CIFAR-10C-LT test datasets, which include label distribution shifts. CIFAR-10C (Hendrycks & Dicttcrich, 2019) serves as benchmark for assessing model robustness against. 15 different predefined lypes of corruptions, such Gaussian noise. To analyze the effects of label distribution shifts, we define the number of samples for class k as n n(1/p)
where is the number of samples for the head class and p is the class imbalance ratio. In Fig. 3 we observe substantial performance drops in test accuracy for BNAdapt (orange) as the class imbalance ratio p increases. The BNAdapt exhibits even worse performance than NoAdapt. (blue) when p = 100.
In Fig. we provide more detailed analysis for misclassification patterns by presenting confusion matrices illustrating the impact of two distinct long-tailed distributions (each row) across four types of image corruptions (each column, including clean, Gaussian noise, Defocus blur, and Snow). These matrices show the fraction of test samples from class i classified into class j. The trends observed in these real experiments are similar to those analyzed in our toy example. Firstly, significant accuracy degradation is evident, especially for head classes (with smaller/late /larger class indices in the 1st/2nd row, respectively). Secondly, the confusion patterns remain consistent across different corruption types and reflect class-wise relationships; for example, more frequent confusion occurs between classes with similar characteristics, such as airplane & ship (0 and 8), automobile & truck (1 and 9), and cat & dog (3 and 5). Additionally, we present confusion matrices for CIFAR-10C-IL with various levels of label distribution shifts (p 1, 10. 100) in Figures g 11 of Appendix F revealing increasingly pronounced class- wise confusion patterns as the imbalance ratio p rises from 1 to 100.
4

---

Published in Transactions on Machine Learning Research (02/2025)
These observations suggest the potential to correct the inaccurate predictions of the BN-adapted classifier by reversing the effects of class-wise confusion. Similar problem has been considered in robust model training with label noise (Natarajan et al. 2013 Patrini et al. 2017 Zhu et al. 2021), where attempts have been made to adjust model outputs using an appropriate affine transformation, reversing the ostimated label noise patterns training dataset. However, a new challenge arises TTA scenarios where only unlabeled test data with unknown (online) label distributions are available, which hinders the accurate estimation of class-wise confusion patterns. In particular, commonly used label correction scheme in robust learning settings (Zhul et al. 2021 fails to estimate the confusion matrix resulting from label distribution shifts in TTA scenarios, shown in Appendix H To address these challenges, we next propose new metric to detect label distribution shifts and correct the inaccurate predictions from BN-adapted classifiers.
A new metric to detect label distribution shifts in TTA During test time, although we only have access to unlabeled lest samples, can obtain pseudo soft labels for these samples using the BN-adapted pre-trained classifier. A common metric for detecting label distribution shifts is the average pseudo- distribution of the test samples, as used in previous studics (Park et al.] 2023 Zhao et al. 2022 Zhou et al. 2023). However, we find that relying solely on this metric is insuflicient for accurately detecting label distribution shifts. To illustrate this, we use the CIFAR-10C-iml dataset (Niu et al.) 2023), which models online label distribution shifts with varying severity.
consists of 10 subsets, each with a class distribution defined by [P1, P2......... PKk] where p& Pmax and p; = Puin = (1 Pmax)/ 1) for i + k. The imbalance ratio (IR) is defined as IR Pmmax/Pmin. Each subset has 1,000 samples, totaling 10,000 for the test set (detailed in Appendix Aj. If the average pseudo-label distribution accurately estimates the shift from a uniform distribution, the distance D(u,Pt, (using cross-entropy) between the uniform distribution u and the average pseudo-label distribution Pø := [ E(.,)EB softmax(fa(=)) calculated for each test batch B using the BN-adapted classifier fg. should increase with IR. However, as shown in Figure 4 (left), D(u, PB) increases from IR 1 (blue points) to 20 (green) but decreases again for more severe shifts (e.g., IR. > 50), even though test accuracy continues to drop. This indicates that the average pseudo-label distribution po alone is inadequate for distinguishing between no shifts (IR 1) and severe shifts (IR 5000). To address this limitation, we introduce a new metric, prediction deviation (from uniformity) d3 TB² Σian)eB D(u, ((u.softmax(ff (z)),, which measures the average confidence of predictions for each batch. Figure 4 (right) shows that as IR increases, both prediction deviation and test accuracy decrease, effectively quantifying the severity of label distribution shifts. Furthermore, we can now see that the decline in D(u, PB) under severe shifts is due to the unconfident predictions, making Ps appear closer to uniform. We use both the average pscudo-label distribution ps and prediction variance dg to detect the magnitude and direction of label distribution shifts.
3 Label Distribution Shift-Aware Prediction Refinement for TTA
We introduce prediction refinement module that can detect test-time class distribution shifts and modify the predictions of the trained classifiers to effectively reverse the class-wise confusion patterns.
Prediction refinement modulc Io Our core idea is that if the prediction refinement module ge experiences batches with diverse class distributions before the test time, it can develop the ability to refine inaccurate predictions resulting from label distribution shifts. Based on this insight, we train 96 during the intermediate time, between the training and testing times, by exposing it to several batches with diverse class distributions from the training datasets. The module Go takes as inputs the average pseudo-label distribution ÎB = [ST E softmax((f(x1)) E K and prediction variance dg- = T&T E(E))EB Du,, softmax each batch, to detect the label distribution shifts from uniformity. The module then outputs a square matrix Wi E R and a vector by E RKK which logether transform the model's logits fa(x)) for a sample x E B to fa(z)WB + bg.
I
e R for
Training of 96 During the intermediate time, we use the labeled training dataset D as the intermediate dataset D Dint. assuming that D is available while the test dataset Diest remains unavailable, as common in previous TTA settings (Choi et al., 2022. Lim et al., 2022; Park et al. 20023). For example, we use CIFAR-10 dataset. during the intermediate time on CIFAR-10C-imb benchmark. To create batches with diverse class
5

---

Published in Transactions on Machine Learning Research (02/2025)
0.8
0.8
Test (96
80
2.66
0.6
0.5
60
NoAdapt
BNAdapt
0.3
0.3
IR20
BNAdapt (ours)
I850
IR5000
10
100
2:30
2.32
2.34
2.36
2.38
4.50 4.75 5.00
5.25
5.50
5.75
6.00
6.25
Class imbalance:
Dlu, St Esstmaxttoooo
2olu, suftmantaaaalll
We observe performance degra- dation of BNAdapt (orange) as the class imbalance ratio p inereases Laile CIFAR-10C. apPT-ap BNAdapt (green) shows consistently improved per- formance regardless of class imbalance.
Figure 3:
We demonstrate the relationship between (left) test accuracy and the difference between averaged pseudo label dis- tribution, [8] Ex,es soflmax([[(x:)) and uniform distribution, u, and (right) the test accuracy and the prediction deviation ds, for each batch with the BN-adapted classifier for CIFAR- 10C-imb under Gaussian noise of different imbalance ratios. Each point represents single batch within the test dataset.
Figure 4:
distributions during the intermediate time, we employ a Dirichlet distribution (Gong et al. 2022 Yurochkin el al. 2019). Batches sampled through i..d. sampling lypically have class distributions resembling a uniform distribution. In contrast, batches sampled using the Dirichlet distribution exhibit wide range of class distributions, including long-tailed distributions, illustrated in Fig. of Appendix A If the original training dataset exhibits imbalanced class distribution, seen in datasets like PACS and OfficeHomn (Fig. 5 this imbalance can inadvertently influence the class distributions within batches. To mitigate this, we create a balanced intermediate dataset Ditt by uniformly sampling data from each class.
The training objective 96 for a Dirichlet-sampled batch Bir C Dint formulated as follows:
Cimb(d) (a..)) [CE(y, softmax(fa( Wimb (Ø) + bamb
(1)
for (omb(o), (imb(() = go (Panb, dimb) where Pimb is the averaged pscudo label distribution, dmmb is the prediction deviation of the batch BDir and CE denotes the cross-entropy loss. During the intermediate time, 9o is optimized to minimize the CE loss between the ground truth labels of the training samples and the modified softmax probability. During this time, the parameters of the trained classifier fo are not updated, but the batch statistics in the classifiers are updated as in BNAdapt. Moreover, since 9s training is independent of the test domains, e.g. test corruption types, we train Io only once for each classifier. Therefore, it requires negligible computation overhead (Appendix A).
Regularization for 9o When there is no label distribution shift, there are no significant class-wits wise confusion patterns that the module needs to reverse (as shown in Fig. 9h. However, when using only equation [ the module may become biased lowards class-inbalanced situations due to exposure to batches with inbalanced class configurations from Dirichlet sampling. To prevent this bias, we add regularization term the training objective of Ja, encouraging it to produce outputs similar to an identity matrix and zero vector when exposed to batches i.i.d. sampled from the class balanced intermediate dataset Dint The regularization for an i..dd sampled batch BuD C Dint. is formulated as
(o) E(,,,)
[xkk) MSE (0), 0k)]
(2)
for Wha(()) bust (6) = Io (Phat, dhat).
Overall objective Combining these two losses, we propose a training objective for go
((d) Limb (0) + (Lbaa()),
(3)

---

Published in Transactions on Machine Learning Research (02/2025)
where is hyperparameter that balances the two losses for class-imbuns class-imbalanced and balanced intermediate batches. We simply set a to 0.1 unless specified. The change in the outputs of go with respect to different a The pseudocode for DART' is presented in Appendix B
can be observed in Appendix F
Utilizing 96 at test-time Since the prediction refinement module Is only requires the averaged pseudo label distribution and the prediction deviation as inputs, it can be employed at test time when only unlabeled test data is available. To generate the affine transformation for prediction refinement, 96 uses the outputs of the BN-adapted pre-trained classifier Sā, not the classifier So that is continually updated during test. time by some TTA method. This is because g6 has been trained to correct the inaccurate predictions of the BN-adapted classifier caused by label distribution shifts during the intermediate time. Specifically, 9 generates WB and by for prediction refinement of a test batch B:
WB, by 9o (PB, dø) where ps
1 BB
Σ SEB
softmax(Jo, (8)), dB
1 B
Σ DEB
(oftmmmotxx((ee (8)).
For a test data & E B, we modify the prediction from ʃa from coftmax(fo($))) to softmax (f) + bg)- These modified predictions can then be used for the adaptation of fo- Consequently, our method can be integrated as plug-in with any TTA methods that rely on pseudo labels obtained from the classifier. Details about the integration with existing TTA methods can be found in Appendix A
4 Experiments
Benchmarks We consider two types of input data distribution shifts: synthetic and natural. Synthetic distribution shifts are artificially created using data augmentation techniques, such as image corruption with Gaussian noise. In contrast, natural distribution shifts arise from changes in image style, for instance, from artistic to photographic styles. We evaluate synthetic distribution shifts on CIFAR- 10/R--00/000CC 1000C and ImageNet-C (ondryykss & Dictterich 2019). and natural distribution shifts on CIFAR-10.1 (Recht et al., 2018), PACS (Li eet al. 2017), and OfficeHome (Venkateswara et al. 2017] benchmarks. For syuthetic distribution shifts, we apply 15 different types of common corruption, each the highest severity level (i.e., level 5). To evaluate the impact of class distribution shifts, we use long-tailed distributions for CIFAR-10C (class imbalance ratio p) and online imbalanced distributions for CIFAR-100C and ImageNet-C (imbalance ratio IR). For datasets with large number of classes, such as CIFAR-100C and ImageNet-CC, the test batch size is often smaller than the number of classes, making the distribution of test batches significantly different from the assumed long-tailed distributions. Therefore, to evaluate the impact of class distribution shifts, we consider test batches modeled by online label distuibutio shifts, as described in Sec. 2 (with further details in Sec. A. For PACS and OfficeHome, we use the original datasets, which inherently include natural label distribution shifts across domains (Fig. 5
Basclines Our method can be used a plug-in for baseline methods that utilize test data predictions. We test the efficacy of DART combined with following baselines:
(1) BNAdapt (Schneider et al. 2020)
corrects batch statistics using test data; (2) TENT (Wang et al., (020) fine-tunes BN layer parameters to minimize the prediction entropy of test data; (3) PL [Lee 2013) fine-tunes the trained classifier using confident. pseudo-labeled test samples; (4) NOTE (Gong et al. 2022) adapts classifiers while mitigating the effects of non-i.i.d lest data streams through instance-aware BN and prediction- reservoir sampling; (5) DELTA (Zhao et al., 2022) addresses incorrect BN statistics and prediction bias with test-time batch renormalization and dynamic online rewcighting: (6) ODS (Zhou et al. 2023) estimates test data label distribution via Laplacian-reegllarized maximum likelihood estimation and adapts the model by weighting infrequent and frequent classes differently. (7) LAME (Boudiaf et al.) 2022) modifies predictions with saplacian-regulatte maximum likelihood estimation using nearest neighbor information in the classifier's 's embedding space. (8) SAR (Nu et al. 2023) adapts models to lie in a flat region on the entropy loss surface. More details about baselines are available in Appendix
Experimental details We use 2e (He et al. 2016 for CIFAR and ResNet-50 for PACS, OfficeHome, and ImageNet benchmarks. For the distribution shift-aware module gø, we use 2-layer MT. (Haykin
7

---

Published in Transactions on Machine Learning Research (02/2025)
Table Average accuracy on CIFAR-IOC/LO0L-LI,, PACS, and OfficeHome. Bold indicates the best performance for each benchmark. The prediction refinement of DART consistently contributes to effective adaptation combined with existing entropy minimization-basse TTA methods.
CIFAR 10C-IT
CIFAR-10.1 10.1-IT
Method
PACS
OfficeHoome
== 10
p= 100
p=
p 10
A 100
NoAdupt
71.7:40.0
71.3:0.1
71.440.2
87.7+0.0
87.3
88.340.7
60.7+:0.0
60.8+0000
AdAdapt (Schneider et al.
2020
85.2+0.0
79.0+0.1
67.0+0.1
85.6+0.2
77.8+0.5
64.6+0.6
72.5++0.0
60.4+00.1
BNAcapt+DARI (ours)
85.2+0.1
84.7+0.1
85.1+0.3
85.6+00.
84.5+004
85.6+0.7
76.4++0.1
61.0+0.1
TENT (Wagg et al.
2020
86.3+0.11
82.9:0044
70.4+0.1
85.8+0.3
77.6+1.5
64.6+111
76.8:+0.6
60.9:10.2
TENT +DAR [ours]
86.520022
86.710.3
88.210.33
85.71004
85.010..7
85.810.2
81.61004
62.1100.2
PL (Loe 2013]
86.54-0.1
02.9--0.2
68.14-0.2
86.2-10.6
77.6+114
64.14:1.1
72.44-0.2
59.24:0.2
PL PLADATT (ours)
86.6100.1
86.810.2
86. 86.320.4
86.2:10.4
85.0+1.7
85.411.11
76.71:0.4
60.140044
NOTE (Gong et al.
2022
81.8+0.3
81.1++0.5
79.8+
84.4+HU.7
84,4+-1.2
84.6++2.4
79.1++0.8
53.7++0.3
NOTE-DART (ours)
81.620.4
81.920.5
82, 220.4
$1.0+1.0
85.840.8
87.020.9
79.9::0.3
53.540.11
LAME (Boudiaf et al.
2022
85.2+0.0
80.4::0.1
70. 02..2
85.6:1.44
78.74:1.5
67.2+1100
72.5:10.33
59 59.8:1..1
LAME +AATT (owss
85,2+0.0
82.8:00. 1
81.220.2
85.6+1..5
81.91110
80,720.5
76.0:10.2
60.2:0..2
DELTA (Zhuo et al
2022
85.9:-0.1
82, 02.5:10.66
70.4+-0.3
85.4+-0.2
77.84+3.1
64.2:12999
78. 78.24400.6
63.14-0.1
DELTA+DART (ours)
85.8+0.2
85.9+0.66
87.6=0.4
85.3+0.3
85.1+:1.2
86.6+0.99
01.7+00.4
63.
ODS Zhou et al.
2023
85,920..1
83.
76.520.4
86.8+00.2
85.3+11.6
81.0:0.9
77.4+0.5
62.1
ODS-DART (ours)
85.9200.1
85.0=0.4
84.9=0.2
86.7+:03
87.4=1.4
88.
79.7#:0.4
62.5+0022
SAR NNu 2023
86.810.1
81.0 0.3
68. 2:0.2
86.310.4
77.7+1.5
61.410.9
71.21004
61.11000
SAR+DARI [nurs)
86. 66.71+0.1
86.4+0.3
87.0++0.4
86.14-0.3
84.9+HU.8
85.6+00.4
78.5++0.2
61.7+00.2
with a hidden dimension of 1,000. Fine-Luning layers, optimizers, and hyperparameters remain consistent with those introduced in each baseline for fair comparison. Implementation details for pre-training, intermediate-time training, and test-time adaptation are described in Appendix A.
1998)
4.1 Experimental Results
DART-applied TTA methods In Table we compare the experimental results for the original vs. DART- applied TTT methods across CIFAR-10C/10.11 enhances the performance of existing TTA methods for both synthetic and natural distribution shifts. In particular, DART achieves significant gains as the class imbalance ratio P increases, while maintaining the original performance for = 1. For instance, on CIFAR-10C-LT with class imbalance ratios 10 and 100, DART improves the test accuracy of BNAdapt by 5.7% and 18.1%, respectively. DART's performance improvement can vary depending on how the baseline method handles pseudo labels. The baseline methods discussed in this paper can be categorized into two categorics: those designed mitigate the effocts of label distribution shift, such NOTE, ODS, and DELTA, and the remaining baseline methods like TENT and SAR. As shown in Table 1 methods like TENT and SAR suffer significant performance degradation due to label distribution shifts because they do not consider the presence intensity of test-time label distribution shifts. And, DART successfully recovers the predictions significantly degraded by label distribution shift. Conversely, methods such as NOTE, which builds a prediction balancing memory for adaptation, and DELTA and ODS, which compute class frequencies in the test data and assign higher weights to infrequent class data, show less performance degradation by label distribution shifts compared to other baseline methods. However, since these methods still rely on pseudo-labeled test data for memory construction or weight computation, the accuracy of pscudo labels significantly impacts their performance. Thus, observed in Table I DART's prediction refinement scheme also contributes to performance improvement in these methods under label distribution shifts. In conclusion, the degree of performance degradation due to label distribution shifts and performance recovery by DART varies depending on the baseline methods used.
PACS and OfficeHome benchmarks. DART consistently
DART on large-scale benchmarks To apply DART to sarger-scale benchmarks, such as CIFAR-100C-inll and ImageNet-C-inll we introduce variant called DART-split. The main difference is that DART-split divides the prediction refinement module IS into two parts: Jø, for detecting the severity of label distribution shifts and go, for generating an effective affine transformation for prediction refinement. Originally, DART
8

---

Published in Transactions on Machine Learning Research (02/2025)
Table 2: Average accuracy on CIFAR-1OOC, and ImageNet-C under online imbalance setup with several imbalance ratios (IR). Both DART and spRT- effectively correct inaccurate predictions. On the large- scale benchmarks such as CIFAR-100C and ImageNet-C, DART-split demonstrates superior performance. Refer to Table X for the combined results with other TTA methods.
CIFAR-100C-imb
ImageNet- C-imb
IR1
IR200
IR500
IR50000
IRI
IR1000
IR5000
IR500000
NoAdapt
41.5=+0.3
41.5+0.3
41.4+03
41.5+0.3
18.0+0.0
18.0#0.1
18.0#+.0
18.1#0.1
BNAdapt
59.810.55
29.1100.5
18.810.4
9.310.1
31,510.00
19.810.1
8,410.1
3.410.0
BNAdapt+ DART (ours)
59.910.33
41.810.8
32.210.7
22.7110.6
BNAdapt+ DART-split (ours)
59.7.00.5
45.810.2
50.210.3
49.110.77
31,510.0
20.020.2
11.520.6
8.210.5
Table 3: Ablation studies highlighting the critical role of the prediction refinement module's design, particularly the significance of its inputs and outputs, in enhancing DART's effectiveness.
Do input. dB
Po output
CIFAR-10C-LT
W
p= 10
p= 100
ANAdapt
85.210.0
79.0
67.0
V
V
81.8
78.0+11.0
81.9
85.0+0.11
84.0=1.22
83.0
BNAd (ours)
85.4+0.0
83.6#0.2
80.7+0.1
85.2#0.1
84.7#0.1
85.1+0.3
used single module So to handle both roles, trained with the loss equation 1 for refining predictions on diverse class-iballaceed batches, regularized by equation 2 using nearly class-balancod batches. However, with large number of classes, the module needs to experience numerous types of distribution shifts during intermediate training to generate effective transformations at test time, regardless of the severity of class distributtinn shifts. To address this, DART-split separates these tasks. Iø, is trained exclusively on Dirichlet-sampled batches, focusing on severely class-imbalanced batches for prediction refinement. The entire module then decides whether to apply goz 's refinement for each Lest. batch based on the output of gør, which detects the severity of label distribution shifts using the prediction deviation dg of each batch. Iø, outputs a severity score sg ranging from 1, indicating the degroe of label distribution shift. If sg exceeds 0.5, indicating a severe shift, the batch's 's predictions are modified using the alline transformation WB and by generated by 962s i... softmax(fo(r)Ws + bs). Otherwise, the prediction is not modified. More detailed explanation of DART-split is provided in Appendix B In Table 2 compare the experimental results for the original vs. DART-split-ppplied BNAdapt on the CIFAR- 100--bb and ImageNet-C-inn benchmarks. DART-split consistently improves performance, achieving a 16.7-39.80 increase on CIFAR-100C-imb with imbalance ratios of 200-50000 and a 0.2-4.8% increase on ImageNet-C C-imb with imbalance ratios of 1000-500000. It notably outperforms BNAdapt under severe label distribution shifts and surpasses NoAAapt by over 4% across all imbalance ratios on CIFAR-L00C-imb, In Table we also show the results of DART-split combined with existing TTA methods demonstrating consistent performance gains, CIFAR-100C-imb and ImageNet-C-inm Additionally, report the performance of DART-split compared to DART on CIFAR-10C-LT in Table 7
4.2 Ablation Studies
Importance of inputs and outputs of g6 for label distribution detection and prediction refinement In Table 3 we present ablation studies demonstrating the effectiveness of the two inputs-the averaged pscudo- label distribution P8 and prediction deviation dg and the two outputs, W E R and E R of DART in prediction refinement. First, we observe that using the prediction deviation dg as an input, in addition to the average pscudo-label, results in gain of 2.2-6.7% when do outputs both W&b. Additionally, using the square matrix W for prediction refinement outperforms using the bias vector as label distribution shifts become more severe, showing improvements of 0.4% and 2.3% on COCAR-10C-LT with p 10 and 100, respectively.

---

Published in Transactions on Machine Learning Research (02/2025)
The best performance for p 10 and 100 is achieved when both W and b are used. These results align with the analysis in Section 2 indicating that as label distribution shifts become more severe, class-wise confusion patterns become clearer, making it crucial to reverse these patterns for improving BN BNAddpt's performance.
Table 4: Average accuracy (%) CIFAR-10C and CIFAR-10C-imb of DART-applied BNAdapt with different sampling methods during intermediate time.
Sampling int, time
CIFAR-10C-LT
CIFAR-10C-imb
Dirichelt (used) Unt&LT(p 20)
p
p= 10
p= 100
IR1
IR20
IR50
IR5000
BNAdapt
85.240.0
79.0#0.1
67.0+b011
85,3+0.1
50.0#0.1
31.60.4
20.3+0.1
65.2100.1
84.7100.1
85.110.3
85.310.2
76.210.3
78.210.1
82.410.7
BNAdapt+DDRR (ours)
85.4.10.0
85.6.10.1
87.3.10.2
85.5.10.11
58.910.55
43.81.0.6
28.7.00.2
Importance of Dirichlet sampling during intermediate time We use Dirichlet sampling during the intermediate time to make the prediction refinement module experience intermediate batches with diverse label distributions. To evaluate the effectiveness of the Dirichlet sampling, we consider a scenario where the module experiences only three types of batches during the intermediate time: uniform, long-tailed with a class imbalance ratio p = 20, and inversely long-tailed class distributions, inspired by Park et al. (2233 In Table 4 we report the performance of ANAdapt combined with this DART variant on CIFAR- 1OC-LL and CIFAR-10C-imb. When experiencing only uniform and long-tailed distributions during the intermediate time, the test accuracy of the variant of DART surpasses that of the original DART on CIFAR-10C-LI since it experiences only similar label distribution shifts during the intermediate time. However, on CIFAR-10C- imb, we can observe that the DART variant's 's performance is significantly lower since the Lest-lime label distributions become more diverse and severely imbalanced. Thus, using Dirichlet sampling to experience a wide range of class distributions during the intermediate time is crucial for effectively addressing diverse test time label distribution shifts.
Table 5: Average accuracy (%) on CIFAR-1OC-LT of DART with different Is outputs.
96 output
CIFAR-10C-LT
affine (used) additive params
p= 1
p= 10
p= ==
BNAdapt
85.2+0.0
79.0+0.1
67.0+0.1
85.2+0.1
84.740.1
85.1+0.3
R (ours)
82.0+2.2
79.942.0
1..
Importance of DART prediction refinement scheme using affine transformation To Lest the effectiveness of the DART's prediction refinement scheme using the affine transformation, we consider a variant of DART that modifics Ie to generate the additive paramcters for a part of the classifier, including affine parameters for the output of the feature extractor and the weight difference for the last linear classifier weights, inspired by T.SA (Park et al.] 2023). For this case, the output dimension of 9o gets larger since it is proportional to the feature dimension of the trained model (e.g. 512 for ResNet26). For a fair comparison, we maintain the number of parameters of go- In Table 5 we report the performance of BNAdapt combined with the DART variant on CIFAR-1OC-ITT. The variant DART shows worse performance on all ps and significant performance variability depending on different random scods. We conjecture that this is because Io struggles to learn to generate additive parameters proportional to the feature dimensions in response to various class distribution shifts.
We also present additional experiments in Appendix G These experiments demonstrate the effectivenoss of DART in a more challenging TTA scenario, where Lest samples with multiple types of corruption are encountered simultaneously. We also verify the robustness of DART against changes in hyperparameters, such as the structure of 96 and intermediate batch size. Morcover, we introduce a variant of DART that reduces the dependency on labeled training data during the intermediate training phase of DART by utilizing the condensed training dataset in Kang et al. (2023
10

---

Published in Transactions on Machine Learning Research (02/2025)
5 Related Works
Methods to handle label distribution shifts The scenario where class distributions differ between training and test datasets has been primarily studied in the context of long-tail learning. Long-tail learning
addresses class imbalance through two main approaches: re-sampling (Chawla et al. 2002 Liu et al. 2008) and
re-weighting (Hong et al., 2021 Menon et al. 2020). Recently, the impact of test-time label distribution shift in domain adaptation has gained attention. Specifically, RLSbench (Garg et al. 2023) reveals performance
drops of TTA methods, BNAdapt and TENT, under label distribution shifts, and proposes a recovery method through re-sampling and re-weighting. However, the method is unsuitable for TTA settings due to the simultaneous use of training and test data. DART' enhances the BN-adapted classifier predictions by correcting errors through a prediction refinement module, without relying on training data during test time.
TTA method utilizing intermediate time Recent works CCoo et al. 2022 Lim et al., 2022 have explored methods to prepare for unknown test-time distribution shifts by leveraging the training dataset. during an intermediate time. For instance, LSA (Park et. al., 2023) conducts intermediate (raining to enhance classifier performance distributions. Specifically, LSA trains a label shift adapter using batches with three types of class distributions (uniform, imbalanced training distribution, and inversely imbalanced) to produce additive parameters for the classifier In contrast, DART exposes the prediction refinement module to a more diverse range of class distributions through Dirichlet sampling and generates an effective affine transformation for logit refinement, Additionally, while LSA uses only averaged pseudo-label distributions to estimate label distribution shifts, DART also considers prediction deviation to more effectively detect label distribution shifts. In Section G we demonstrate the efficacy of DART's sampling and prediction modification schemes compared to LSA. More related works are reviewed in Appendix C
baig-tilldd training datasets when applied to test datasets with different long-tailed
6
Discussion and Conclusion
6.1 Discussion
Performance on large-scale benchmarks While DART consistently enhances BN-adapted classifiers on large-scale datasets such OfficeHome and PACS under test-time label distribution shifts (Table 1 the performance gains by DART may appear less noticcable large-scale benchmarks compared to small-scale benchmarks. In these settings, the batch size is typically smaller than the total number of classes, so even if the test dataset's s class distribution differs significantly from that of the training dataset, each batch experiences only minimal distribution shifts. Consequently, while DART's improvement may seem less significant compared to those on small-scale datasets like CIFAR-1OC, it still effectively adjusts inaccurate predictions by label distribution shifts. Additionally, as the number of classes increases, the variety of potential class distribution shifts grows dramatically. Therefore, training the prediction refinement module at the intermediate time becomes more challenging on large-scale benchmarks with many classes, such TmageNet--C (Table 2 This highlights need for further research on more efficient methods to train the prediction refinement module, especially in extremely large label spaces.
The use of training data during the intermediate time Although DART employs labelod training data to train its prediction refinement module before the test time, it does not violate TTA protocols and is commonly adopted by various TTA methods. In particular, DART's intermediate-time training with labeled data does not conflict with TTA guidelines because the training data is exclusively used to refine the prediction module during the intermediate time and is no longer accessible during the test time. Moreover, as described in Section 5 several recent. TTA methods (Choi et al. 2022 Lim et al. 2022 Park et al., 2023) also utilize training data before test time to prepare for unknown distribution shifts. Nevertheless, in scenarios where the labeled training data for a given trained model is unclear, obtaining the training data can be challenging. To increase the scalability of DART in these situations, exploring approaches that leverage an auxiliary dataset instead of the original training data may be promising direction for future work.
11

---

Published in Transactions on Machine Learning Research (02/2025)
6.2 Conclusion
We proposed DART, method to mitigate lest-time class distribution shifts by accounting for class-wise confusion patterns. DART trains a prediction refinement module during an intermediate time to detect label distribution shifts and correct predictions of BN-adapted classificrs. Our experiments demonstrato DART's effectiveness across various benchmarks, including synthetic and natural distribution shifts. DART's internediate-time training incurs additional costs compared traditional TTA. However, these costs are relatively minimal, especially considering the significant performance improvements it offers when combined with traditional TTA methods (Table D The low cost is partly due to using the existing training dataset, rather than requiring auxiliary dalaset, and its short runtime. This paper highlights and elfectively addresses the negative impact of test-time label distribution shifts on existing TTA methods. As label distribution shifts are common in real-world scenarios, we expect future studies to build on our approach to enhance real-world scalability by making models robust to test-time shifts in both input and label distributions.
Broader Impact Statement
This paper highlights and effectively resolves the negative impact of test-time label distribution shifts to existing TTA methods. Since the label distribution shifts between the training and test domains are common in the real world, we expect further studies that are robust to test-time shifts on both input data and label distributions to be built on our work to enhance real-world scalability.
Acknowledgement
This work was supported by the National Research Foundation Korea (NRF) grant funded by the Korea government. (MSIT) (No. RS-2024-00408003 and No. 2021RIC1C1100533))
12

---

Published in Transactions on Machine Learning Research (02/2025)
References
Abien Fred Agarap. Deep learning using rectified linear units (relu). arXiv preprint orXie: (at::18030877, 2018.
Malik Boudiaf, Romain Mueller, Ismail Ben Ayed, and Luca Bertinetto. Parameter-fre online test- test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 8344 8353, 2022.
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei Efros, and Jun- Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pottern Recognition, pp. 11500--79,,00 2022.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-388 357, 2002.
Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IREE/CVE international conference on computer vision, pp. 9640-9649, 2021.
Sungha Choi, Seunghan Yang, Seokeon Choi, and Sungrack Yun. Improving test-time adaptation via shift- agnostic weight regularization and nearest source prototypes. In European Conference on Computer Vision, pp. 440 458. Springer, 2022.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.044 04805, 2018.
Saurabh Garg, Nick Erickson, James Sharpnack, Alex Smola, Sivaraman Balakrishnan, and Zachary Chase Lipton. Risbench: Domain adaptation under relaxed label shift. In International Conference on Machine PMLR, 2023.
Learning, pp.
Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. Note: Robust continual test-time adaptation against temporal correlation. Advances in Newral Information Processing Systems, 35:27253 27266, 2022.
Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Test time adaptation via conjugate pseudo-labels. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Simon Haykin. Neural networks: comprehensive foundation. Prentice Hall PTR, 1998.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recogrétion (CVPR), pp. 770-778, 2016. doi: 2011099/CV2.2006.900
Dan Hendrycks and Thomas Dielterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arxio preprinl 10X.011503.0951,, 2015.
Youngkyu Hong, Seungju Han, Kwanghee Choi, Seokjun Seo, Beomsu Kim, and Buru Chang. Disentangling label distribution for long-tailed visual recognition. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. (626-6636, 2021.
Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems, 34:2427-2440, 2021.
Minguk Jang, Sac- Young Chung, and Hye Won Chung. Test-time adaptation via self self-training with nearest neighbor information. In The Eleveuth International Conference on Learning Representations, 2022.
Juwon Kang, Naycong Kim, Kwon Donghyeon, Jungseul Ok, and Suha Kwak. Leveraging proxy of training data for test-time adaptation. In International Conference Machine Learning (TCML), July 2023.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. or 20x:141.6980 2014.
arXiv preprint
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images, Technical Report 0, University of Toronto, Toronto, Ontario, 2009.
Alex Krizhevsky, Πya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
Dong-Hyun Leo. Pseudo-label: : The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, 2013.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier domain generalization. In IEEE International Conference Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pp. 5543-5551. IEEE Computer Society, 2017. doi: (11109/CCCV.011..50
Hyesu Lim, Byeonggeun Kim, Jaegul Choo, and Sungha Choi. Tin: domain-shift aware batch normalization in test-time adaptation. In The Eleventh International Conference on Learning Representations, 2022.
Xu- Ying Liu, Jianxin Wu, and Chi-Hua Zhou. Exploratory undersampling for class-imbalang learning. 1EEE Transactions on Systems, Man, and Cybernetics, Part (Cybernetics), 39 (9()):539-55)) 2008.
Robert A Marsden, Mario Döbler, and Bin Yang. Universal test-time adaptation through weight ensem- bling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2555-2565. 2024.
Russell Mendonca, Xinyang Geng, Chelsea Finn, and Sergey Levine. Meta-reinforcement learning robust to distributional shift via model identification and experience relabeling. CoRR, abs/ abs/2006.07118, 2020.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. arXiv preprint arXi:2207.009114 2020.
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D' A'mour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006.1000 2020.
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. Advances in neural information processing systems, 26, 2013.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In The Fleventh Internalional Conference Learning Representations, 2023.
Sunghyun Park, Seunghan Yang, Jaegul Choo, and Sungrack Yun. Label shift adapter for test-time adaptation under covariate and label shifts. arXiv preprint arXiv:2308. 08810, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zoming Lin, Natalia Gimelshein, Luca Antiga, ct al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Giorgio Patrini, Alessandro Rozza, Adilya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1944-1952, 2017.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Benjamin Recht, Rebecca Roclofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 10 classifiers generalize to cifar-10? 10? arxiv preprint arXiv:1806 18066004510,, 2018.
14

---

Published in Transactions on Machine Learning Research (02/2025)
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to domains. In Proceedings of the 1 11th European Conference on Compuler Vision: Part IV, ECCV110, pp. 213 Berlin, Heidelberg, 2010. Springer- Verlag. ISBN 364215560X.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustnoss against common corruptions by covariate shift adaptation. Advances in neural information processing systems, 08:11539-11553 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXii::4000 1556, 2014.
Petar Stojanov, Zijian Li, Mingming Gong, Ruichu Cai, Jaime Carbonell, and Kun Zhang. Domain adaptation with invariant representation learning: What transformations to learn? Advances in Neural Information Processing Systems, 34:24791 24803, 2021.
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt, Measuring robustness to natural distribution shifts in image classification. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Newral Information Processing Systems, volume 33, pp. 18583 18599. Curran Associates, Inc., 2020.
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on pallern analysis and machine intelligence, (0(11):1958-1970) 2008.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Hemanth Venkateswara, Jose Euscbio, Shayok Chakraborty, and Sethuraman Panchanathan, Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Compuler Vision and Pattern Recognition, pp. 4018-5022 2017.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2020.
Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise lcarning? Advances in neural information processing systems, 32, 2019.
Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. Dual t: Reducing estimation error for transition matrix in label-noise learning. Advances in neural information processing systems, 33:7260 7271, 2020.
Li Yi, Gezheng Xu, Pengcheng Xu, Jiaqi Li, Ruizhi Pu, Charles Ling, A lan McLeod, and Boyu Wang. When source-free domain adaptation meets lcarning with noisy labels. arXiv preprint arXiv:2301. 13381, 2023.
Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. In International conference on machine learning, pp. 7252-72611 PMTR, 2019.
Yixin Zhang, Zilei Wang, and Weinan He. Class relationship embedded learning for source-free unsupervised domain adaptation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 7619-7629, 2023.
Bowen Zhao, Chen Chen, and Shu-Tao Xia. Delta: Degradation-free fully test-time adaptation. In The Eleventh International Conference on Learning Representations, 2022.
Hao Zhao, Yuejiang Liu, Alexandre Alahi, and Tao Lin. On pitfalls of test-time adaptation. In International Conference on Machine Learning (ICMI.), 2023.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Zhi Zhou, Lan-Zhe Guo, Lin-Han Jia, Dingchu Zhang, and Yu-Feng Li. Ods: test-time adaptation in the presence of open-world data shift. In International Conference on Machine Learning. PMLR, 2023.
Zhaowei Zhu, Yiwen Song, and Yang Liu. Clusterability as an alternative to anchor points when learning with noisy labels. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 12912-12923, PMLR, 18 24 Jul 2021.
16

---

Published in Transactions on Machine Learning Research (02/2025)
Appendix
A Implementation Details
A.1 Details about Dataset
We consider two types of input data distribution shifts: synthetic and natural distribution shifts. The synthetic and natural distribution shifts differ in their generation process. Synthetic distribution shift is artificially generated by data augmentation schemes including image corruption like Gaussian noise and glass blur. On the other hand, the natural distribution shift occurs due to changes in image style transfer, for example, the domain is shifted from artistic to photographic styles.
For the synthetic distribution shift, we test on CTFAR-10/T00C and ImageNet--C, which is created by applying 15 types of common image corruptions (e.g. Gaussian noise and impulse noise) to the clean CIFAR-10/ test. and ImageNet validation datasets. We test on the highest severity (i.e., level-5). CIFAR-10/ 10000C is composed of 10,000 generic images of size 32 by 32 from 10/100 classes, respectively. ImageNet-C is composed of 50,000 generic images of size 224 by 224 from 1,000 classos. The class distributions of the original CIFAR-10, 10/100C 100C and ImageNet-C are balanced. Thus, we consider two types of new test datasets to change the label distributions between training and test. domains. First,, we consider CIFAR-10-LT, which has long-tailed class distributions, as described in Section 2 We set the number of images per class to decrease exponentially as the class index increases. Specifically, we set the number of samples for class i as na n11 m(1/p)+/(K-I)) where n and p denote the number of samples of class 0 and the class imbalance ratio, respectively. We also consider the test set of inversely taile distribution in Figure [I Specifically, we set the number of samples for class k as n& = n(1/))(K-1<b)/(k of class K 1 and the class imbalance ratio, respectively. However, unlike CIFAR- CIFAR-100C-IT and ImageNet-C-I.T tend to not have imbalanced class distributions, since the test. batch size (e.g., 32 or 64) is set to be smaller than the number of classes (100 and 1,000). Thus, we also consider CIFAR-100C-imb and ImageNeel-C-ini C-imb,, whose label distributions keep changing during lest time, as described in Section 2 These datasets are composed of K subsets, where K is the number of classes. We assume a class distribution of the k-th subset as PP......... Px], where pk = Pmax and p. = Pmin (1 Pmax)/((K 1) for i z k. Let IR =R=po Pmax//min represent the imbalance ratio. Each subset consists of 100 samples from the CIFAR-100C and ImageNet-C test set based on the above class distribution. Thus, the new test set for CIFAR-100C/Inan (00000///mmaeee-CCCCCC is composed of 10,000/ 00,000/100,000 samples, respectively. Additionally, we shuffle the subsets to prevent predictions based on their order.
where and p denote the nunber of samples
each test batch of
For the natural distribution shift, we test on 1I.,R-10.1-TTT, PACS, and OfficeHome benchmarks. CIFAR-10. (Recht et al. 2018) is a newly collected test dataset for CIFAR- 10 from the Tinylmages dataset (Torralba et al. 2008). and is known to exhibit a distribution shift from CIFAR-10 due to differences in data collection process and timing. Since the CIFAR-10.1 has a balanced class distribution, we construct a test set having a long-tailed class distribution, named CIFAR-10.1-LT, similar to CIFAR-IOC-LT. PACS benchmark consists of samples from seven classes including dogs and elephants in four domains: photo, art, cartoon, and sketch. In PACS, we test the robustness of classifiers across 12 different scenarios, each using the four domains as training and test domains, respectively. The OfficeHome (Venkateswara et al. 2017) benchmark is one of the well-known large-scale domain adaptation benchmarks, comprising images from four domains (art, clipart, product, and the real world). Each domain consists of images belonging to 65 different categories. The number of data samples per class ranges from a minimum of 15 to a maximum of 99, with an average of 70. This ensures that the label distribution differences between domains are not substantial, as depicted in Figure 5 The data conerattioncoollcttii process of PACS and OfficeHome benchmarks is different across domains, resulting in differently imbalanced class distribution, as illustrated in Figure
A.2 Details about Pre-training
We use Re for CIFAR datasets, and ResNet-50 for PACS, OfficeHome, and ImageNet benchmarks as backbone networks. We use publicly released trained models and codes for fair comparison. Specifically,
17

---

Published in Transactions on Machine Learning Research (02/2023)
PACS deetch
Clsss index Offichhome-art
Cbes hndcx
Obes hdcc
Õas Index
Onicattome-ctpat
Offcertome proddcc
Offcartome realverts
Cless ombbitty
ombolity
Cloce
Class InSRe
Iriees
Class Brdee
Clack (rdlec
Figure 5: Class distribution of PACS and OfficeHlome
for CIFAR-10/100 we train the model with 200 epochs, batch size 200, SGD optimizer, learning rate 0.1, momentum 0.9, and weight decay 0.0005. For PACS and OfficeHome, released pre-trained models of 2023 For ImageNet-C,, C, use the released pre-trained models in the PyTorch library described in (Niu et al. 2023
TTAB Zhao et al.
Paszke et al. 2019
A.3 Details about Intermediate Time Training
We use the labeled dataset in the training domain to train the prediction refinement module Jo for all benchmarks. the original training datasel exhibits an imbalanced class distribution, e.g. PACS and OfficeHome, this imbalance can unintentionally influence the class distributions within the intermediate batches. To mitigate this, we create dlass-balance dataset Dint by uniformly sampling data from each class. For example, for the ImageNet benchmark, the intermediate dataset is subset of the ImageNet training dataset, composed of 50 samples randomly selected from each of the classes. For the intermediate dataset, we use a simple augmentation such as normalization and center cropping.
We use a 2-layer MLP (Haykin 1998) for the prediction refinement module IG. 9o is composed two fully connected layers and ReLU (Agarap) 2018). The hidden dimension of the prediction refinement module is set to 1,000. During the intermediate time, we train 9o by exposing to several batches with diverse class distributions using the labeled training dataset. We train Jo with Adam optimizer (Kingma & Ba 2014), learning rate of 0.001, and cosine annealing for 50 epochs. For large-scale benchmarks, such CIFAR-100 and ImageNet, 9s is trained for 100/200 epochs to expose it to diverse label distribution shifts of many classes, respectively. To make intermediate batches having diverse class distributions, we use Dirichlet sampling as illustrated in Figure 0 and 7 with two hyperparameters, the Dirichlet sampling concentration parameter 6, and the number of chunks Nair- As these two hyperparameters increase, the class distributions of intermediate batches become similar to the uniform. 5 is set 0.001 for ImageNet-C,, 0.1 for CIFAR-100, and 10 for other benchmarks. Nair is set 2000 for ImageNet-C,, 1000 for CIFAR-100C, and 250 for other benchmarks. The intermediate batch size is set to 32/64 for ImageNet and all other benchmarks, respectively. On the other hand, we use regularization on Is to produce W and b that are similar to the identity matrix of size and the zero vector, respectively, for the case when there are no label distribution shifts. The hyperparameter for the regularization a in equation 3 is generally set to 0.1. For PACS and OfficeHome, we set a to 10.
http:://ptthbb.c///ouuaaa//// http://ccuuabbtttttt..ootgttttt https://githubb.cmm/llleee
18

---

Published in Transactions on Machine Learning Research (02/2025)
Dirichlet sampling
i..dd sampling
p=
0=
642
p= 10
0.59
0.2
Olai2
Figure Example of the Dirichlet distribution sampling. IID (i.i.d)) sampling denotes standard uniform sampling. The black dots indicate the class distribution of the sampled batches. The red, blue, and yellow dots represent the class distributions of different class imbalance ratios p, namely 1,2, and 10, respectively. By employing the Dirichlet distribution for batch sampling, we can expose the model numerous batches with diverse class distributions during the intermediate time, thereby enabling it to learn how to mitigate performance degradation caused by class distribution shifts.
20
Cless
Figure 7: Class distributions of five batches sampled from CIFAR-10 10 using Dirichlet sampling. These sampled batches demonstrate diverse and varied class distributions, highlighting the effectiveness of Dirichlet sampling in generating a wide range of label distribution shifts.
A.4 Details about Test- Time Adaptation
During test time, we fine-tune only the Batch Normalization (BN) layer parameters is common with most TTA methods. We use the Adam optimizer with learning rate of 0.001 for all TTA methods in every experiment, except on ImageNet benchmarks, following the approach in TENT (Wang et al. 2020). On ImageNet benchmarks, we use the SGD optimizer with a learning rate of 0.0005. We set the test batch size to 64 for PACS, OfficeHome, and ImageNet-C, and to 200 for CIFAR benchmarks. We run on different random seeds for the intermediate-tinas and test-time training (0,1,2, and 3).
To generate the affine transformation for prediction refinement, Is uses the outputs of the BN-adapted pre-trained classifier Seg: not the classifier fo that continually updated during test time. This is because go has been trained to correct the inaccurate predictions of the BN-adapted classifier caused by label distribution shifts during the intermediate time. Specifically, for test batch B,
WB, by go dg) where PB
Σ softmax(fa (F)), dg REB
Σ D(softmax((, fa, (f))).
(4)
B
[
For a lest data & E B, we modily the prediction from fo from softmax(fo($)) to soflmax(fo($)Ws + bz).
These modified predictions can then be used for the adaptation of fo. Since the affine transforma- tion by Io focuses on prediction refinement, the confidence of predictions can be changed from the original predictions, potentially causing negative effects during test-time training (Goyal et al. 2022). Therefore, we use normalization to maintain the confidence level of predictions. For example, in the
19

---

Published in Transactions on Machine Learning Research (02/2025)
case of TENT (Wang et al. 2020 we adapt the classifier fo using training objective (T) Ju 2 (e) bellz ) log softmax((lfo())lla fo(2)Woth
EEEBI- E. softmax((Ifo(@)1
During the test time, there often arises an issue where the [G
label distribution of unseen test data cxhibits unconfidence, making it challenging to apply prediction refinement through ge directly. To address this, we utilize the softmax lemperature scaling. For OfficeHome and CIFAR100, we multiply logits by 2 and 1.1 for test batches for softmax temperature scaling, respectively.
A.5 Details about Baseline Methods
ANAdapt (Schneider et al.) 2020) BNAdapt does not update the parameters in the trained model, but it corrects the BN statistics in the model using the BN statistics computed on the test batches.
TENT (Wang et a 2020 TENT replaces the BN statistics of the trained classifier with the BN statistics computed in each test batch during test time. And, TENT fine-tunes the BN layer parameters to minimize the prediction entropy of the test data.
2013) PL regards the test data with confident predictions as reliable pseudo-labeled data and fine-tunes the BN layer parameters to minimize cross-entropy loss using these pseudo-labeled data. We set the confidence threshold to 0.95 for filtering out test data with unconfident predictions.
PL (Lee
NOTE aims to miligate the negative effects of a non-i.idd stream during test time by instance-aware BN (TABN) and prediction-balaneed reservoir sampling (PBRS). TABN first detects whether a sample is from dittr--diittiittoinn or not, by comparing the instance normalization (LN) and BN statistics for each sample. For in-distribution samples, IABN uses the standard BN statistics, while for out-of-distribution samples, it corrects the BN statistics using the IN statistics. We set. the hyperparameter to determine the level of detecting aut-of-distribution samples to 4 as used in NOTE (Gong et al. 2022). Due to the non-i.i.d stream, class distribution within each batch is highly imbalanced. Thus, PBRS stores an equal number of predicted test data for each class and does test-time adaptation using the stored data in memory, We set the memory size same as the batch size, for example, 200 for CIFAR benchmarks. NOTE create predictiionballnnee batches and utilize them for adaptation. Thus, the batches for adaptation in NOTE have different class distributions from the test dataset, unlike other baselines including TENT. Therefore, in NOTE, DART used exclusively enhance the prediction accuracy of the examples stored in memory. Since NOTE utilizes IABN layers instead of BN layers, replace the BN layers of the pro-trained model with IABN layers and fine-tune the model for 30 epochs using the labeled training dalaset as described in Gong et al. (2022).
NOTE (Gong et al.
2022)
DELTA ZZhao et al.) 2022) DELTA aims to alleviate the negative effects such as wrong BN statistics and prediction bias by rest-time batch renormalization (TBR) and dynamic online reweighting (DOT). Since the BN statistics computed in the test batch are mostly inaccurate, TBR corrects the BN statisties with renormalization using test-time moving averaged BN statistics with factor of 0.95. DOT computes the class prediction frequency in exponential moving average with factor of 0.95 during test time and uses the estimated class prediction frequency to assign low /oigh weights to frequent /innreiuuennnnnn classes, respectively.
LAME (Boudiaf et. al., 2022) LAME modifies the prediction by Laplacian regularized maximum likelihood estimation considering nearest neighbor information in the embedding space of the trained classifier. Note that LAME does not update network parameters of the classifiers. We compute the similarity among samples for the nearest neighbor information ion with k-NN with k 5
ODS ZZhou et al., 2023) ODS consists of two modules: distribution tracker and a prediction optimizer. The distribution tracker estimates the label distribution w and modified label distribution 2 by LAME, leveraging features of adapted classifier fo and predictions from a pro-trained classifier fe,- The prediction optimizer produces refined predictions by taking an average of the modified label distribution z and the
20

---

Published in Transactions on Machine Learning Research (02/2025)
prediction of the adapted classifier fo. which is trained by weighted loss that assigns high/low weights infrequuett/rrqqeeen classes, respectively.
Since DELTA and ODS utilize predictions to estimate label distribution shifts, our prediction refinement scheme can enhance the label distribution shift estimation, leading to more effective test-time adaptation.
SAR (Niu et al., 2023) SAR adapts the trained models to lie in a flat region on the entropy loss surface using the test samples for which the entropy minimization loss does not exceed 0.4*log(numbb) of classos). We only reset the model for the experiments of online imbalance when the exponential moving average (EMA) of the 2nd loss is smaller than 0.2. The authors of SAR observe significant performance drop when label distribution shifts or batch sizes decrease on BN Adapt,,, explain this drop by the BN layer's property that uses the test batch statistics as shown in Section 2 Thus, they recommend the usage of other normalization layers instead of BN layers. However, in this paper, we analyze the error patterns of BN-adapted classifiers under label distribution shifts and propose a new method to refine the inaccurate prediction of the BN-adapted classifiers. We expect that it can greatly enhance the scalability of BN-adapted classifiers to real-world problems with shifted label distributions.
NOTE, ODS, and DELTA, which address class imbalances by re- re-weighting, the prediction- balancing memory, and the modified BN layers, can be combined with any entropy minimization test-time adaptation methods like TENT, PL, and SAR. For a fair comparison, we report the results of combining these methods with TENT, specifically NOOT,,OTE, TENT+ODS, and TENT+DELTA, in Table D
Table 6: # of parameters, FLOPs, wall clock at the test time. DART-applied BN Adapt does not require additional cost except the memory for 9a- On the other hand, applying DART to other TTA mothods requires more computation costs due to the need for an additional BN-adapted classifier fø, to compute affine transformations. However, the additional cost. caused by Sa, is negligible compared to the performance increases.
Method
# params (# trainable params)
FLOPs
Wall clock
NoAdapt
17.4 M (0 M)
0.86 G
2.24 sec
BN Adapt
17.4 M (0 M)
0.86 G
2.36 see
ADARR
17.6 M (0 M)
0.86 G
2.35 sec
TENT
17.4 M (0.013 M)
0.86 G
3.93 sec
TDAT+
35.0 M (0.013 M)
1.72 G
4.81 sec
A.6 Additional Costs of DART
The additional cost. incurred by DART during intermediate and test times is minimal, while it brings significant performance gains. DART's intermediate time training has a very low computation overhead. This is because the prediction refinement module Is is trained independently of the test domains, requiring training only once for each classifier. The parameters of the pre- classifier, except for BN statistics, are kept frozen while a shallow 96 (a 2-layer ML.P) is trained. For instance, training Je during the intermediate time for CIFAR-10 takes only minutes and seconds on RTX A100. Moreover, the DART-applied TTA methods also do not require significant additional costs compared to naive TTA methods. We summarize the mumber of parameters, FLOPs, and wall clocks on balanced CIFAR-10C in Table 0 When applying DART to BNAdapt, there is almost no additional cost aside from the memory for 96. This is because Is remains fixed during the test time and is shallow. On the other hand, applying DART to other TTA methods requires additional costs due to the need for an additional classifier fa, to compute affine transformations, aside from the continuously adapted classifier. The additional costs by the classifier are insignificant since its parameters are kept frozen during the test time. The small additional cost is negligible (1-2 seconds) compared to the performance gains of 5.7/18. 1% for CIFAR-10C-LT with p 10/100, respectively.
21

---

Published in Transactions on Machine Learning Research (02/2025)
B Comparison of DART and DART-split
Algorithm 1: Intermediate time training of DART
Input: pre-trained classifier fu, labaled training dataset D, hyperparanater for balancing losses, learning
class-balenced training
intereeiaate
Initialize 2-layer MLP for logit refinement
(1) Class-inbalanced internadiate batch
Sample a batch Buu = (Xua, Yus) of size M fron Put using Dirichlet distribation statiatics using the
t B8Adapt(O, BuLe) Adapt BN ho fa(Xin)) 3M loggit of clase-inbalanced internediate batch
batch
(on) bina) <- LogisModifier-DARTY
S6) (Algorithn 2
(o) PPoLe E; CE(Yua.., softaarsss... (a)(d) (d) + bua (+1)
(2) Class-balance internediata batch
Sampla
BID (Xua: Yua) size 1.1.4. fron Ditt + (KAdapt(O, Brio) Adept BB layer statistica using the current batch Jā(Xaa)) MMX logit of clase-balanced internediate batch Whn, daat + LLogitNoodiiirrDAAA (hat, (6) (Algorithm
(o) SSE(Wa (o). [x) HSB(haa())
Update d2
(Coo (d))
Dutpot: trained
Algorithm 2: LogitMModiier--DRRTYY
Input: BK ad-pted claseifier's logit is for current batch 8, prediction refinement module +- softaax(ix) é MMX # softmax output
Computo an averaged pseudo label distribution f R and prodiction deviation dy E R
Ps [8] Σ, P. E R
dy [NT Σ. Diu,pi) R, where an uniforn distribution of size K.
W R RXX and e X reeding pa and de
Ws. by + (Pe
Output: Ws.b8
Algorithm 3: Intermodiate time training of DART-split
Input: pre-trained classifier fu, labeled training dataset D, learning rate 8n and B2 class-balenced training Initialize of single layer and 2-layer MLP 9øz for logit refinement
internediate
(1) Class-tnbalanced Internediate batch Sample batch Botr = (Xua, Yun) of size M from Dus using Dirichlet distribation < BXddapt(O, Bpr) Adapt BB layer statistics using the current batch
fim fa(Xim) 2M llogit class-inbalanced internediate batch
uniform distribution K. Wes(tr), (12)) L.gtTMoulttee-DDT-pplldd.d 9eg) (Algoritha
Tep.el E, D(u, sottaxx)))))
where
(1z()))) = Boue Σ, CE(Yuo.., 2of Waa(d)) + (oa()))))
(2) blass-balanced. internediata batch
Sampla
BIDD (Xua, You) size 1.1.d. from Daat + (8Adapt(9, Bro) Adept Bll layer statistics using the current batch
hai fa() MM logit of clase-balanced internediate batch
pBral D(u, softmax(los,)))
shere uniform distribution size K.
Lanta (01) BCE(O, alno (dwa))) (s,,, (d),)))),
signoid
binary cross antropy
loss
Update 96
Φ1 di 31 V., Cassert (d1)
Φ2 +- dg 92 V Cus (92) Dutput: trained as (go) Bog)
B.1 Detailed Explanation about DART-split
We propose a variant of DART, named DART-split, for large-scale benchmarks, which splits the prediction refinement module 9N into two parts gé, and Jø.. this setup, 9é, takes the prediction deviation as an input to detect. label distribution shifts, while gø, generates affine transformations using the averaged pseudo
22

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 4: LogiiMoodiee--ART--phii
Input: Bl-aadpted classifier's logit for current batch B, part prediction refinement nodule 962
p f- softmax(in) E 24 # softmax output
Compute an averaged peeudo label distribution P R
Ps TBT Pt E
Obtain W R and b RIXK by feading Pg into 962
WE, by 4- [0g (Pa)
Dutput: Wa.bs
label distribution. Specifically, Ié, takes the prediction deviation of each batch B as an input, and outputs a severity score sg = (ge, (ds)) ranging from 0 to 1, with higher values indicating more severe shifts. On the other hand, 9hg uses the averaged pseudo label distribution Ps Ws and bg, effectively correcting predictions affected by the label distribution shifts. DART-split can address the scaling challenges faced by the original DART as the number of classes, K, increases.
input to produce
alline transformation
At the intermediate time, Io: and 962 are trained individually for detecting label distribution shifts and generating affine transformation, respectively, as detailed in Algorithm 3 Dør is trained to classify batches to either severe label distribution shift (output close 1) or no label distribution shift (output close to 0), taking the prediction deviation an imput. Specifically, 96, is trained to minimize BCE(O, (96, (dba))) + BCF(1, a (ge, (dmmb))) where is a sigmoid function, BCE is the binary cross entropy loss, and duss- dmb are the prediction deviation values of the batches i.i.d. sampled and Dirichlet-sampled, respectively. Dø, is composed of a simple single layer, based on the observation in Fig. El that the prediction deviation almost linearly decreases as the imbalance ratio increases. On the other hand, Jos. is composed of a two-layer MLP and focuses on generating affine transformations solely from the averaged psoudo label distribution, following the training objectives similar those of the original DART's 9o.
During testing, each test batch B is first evaluated by 9o. to determine whether there is a severe label distribution shift or not. If SB o (gs, (dx)) excceds 0.5, indicating a severe label distribution shift, the predictions for B are adjusted using the affine transformations from 942, i.e., for a test data E B, the prediction is modified from softmax(f((x)) to softmax(fo(=)Wz + bg). Otherwise, we do not modify the prediction.
For ImageNet-C. C, the output dimension of gø, increases as the number of classes increases, becoming more challenging to learn and generate a higher-dimensional square matrix W and b for large-scale datasets. To address this challenge, set to 0 and fix all entries of b to 0 for ImageNet benchmarks. Moreover, to alleviale the discrepancy in the softmax prediction confidence between the training and test datasets, we store the confidence of the pre-trained classifier's softmax probabilitics and perform softmax temperature scaling to ensure that the prediction confidence of the pre-trained classifier are maintained for the test dataset for each test corruption for ImageNet-C. C. Specifically, for the pre-trained classifier fa,, training dataset D, and test dataset D Pust. we aim find T' that achieves the following:
modify the prediction refinement module to produce W with all off-diagonal entries
( Immax |max softmax (fe (z))a] = EgwD.w [max
(5)
By Taylor series approximation, we can re-write equation as in Hinton et al. (015))
1 maxk JN (x)x
1 + max Sa, (T)a/T
E(aDDD Ex (z)
EiNDesS
(6)
(1+ (2)x
With the assumption that the sum of logits, Ex So, are constant. zu, sum and ft, sum sum and test data, respectively, we can re-write equation b and compute T as
for any training
1 + E (n. joD [max,, fno (x) 1 EgnDrm [max; fo, (2)x
(7)
K + lee, sum / T Eg-Dius [max feo (+)a pe, sum 1+. fmaxe fa (x)2 R K+
K + [te, sum
(8)
1 14 maxx Sop (x)x
K
23

---

Published in Transactions on Machine Learning Research (02/2025)
where K is the number of classes. We store the average of the maximum logits and the sum of logits of the pre-trained classifier on the training data, and use them to calculate the softmax temperature T at test time. For each test batch, we calculate I" by equation [ for the test batch using the maximum logits and the sum of logits of the test batch using the pre-trained classifier. For the (t + 1)-th test batch, we perform softmax temperature scaling using the average value of T calculated from the first to the (t + 1)-th test batches. Calculating the softmax temperature in equation does not require the labels of the test data, ensuring that it does not violate the test-time adaptation setup.
B.2 Comparison of DART and DSRT-split
Table 7: Test accuracy of DART-applied BNAdapt and DART-spiit-pppled BNAdapt on CIFAR-10C-LT
p=
p= 10
p p= 100
BNAdapt BdAdapt+ (ours)
85.2+0.0
79.0+0.11
67.0=10.1
85.2+0..1
84.7++0.1
85.1+0.3
BNAddppt-DARI-spph (ours)
85.0
78.4+0.2
76.1+0.4
Both DART and DART-split utilize the averaged pseudo label distribution and prediction deviation to detect. label distribution shifts and correct the inaccurate predictions of a BN-adapted classifier due to the label distribution shifts, by considering class-wise confusion patterns. However, they differ in the structure of the prediction refinement module gé. The original DART fed two imputs into single prediction refinement module that detects shifts and generates the appropriate affine transformation for each batch. On the other hand, DART-split divides this module into two distinct modules, each detecting the existence and severity of the label distribution shift and generating the affine transformation. We use the same hyperparameters for DART-split as much as possible to those of DART, but ô for CIFAR-100C is set to 0.01, and the softmax temperature for CIFAR-10C during testing is set to 1.1.
As shown Table 1 and 2. we demonstrate that DART and DART-split show consistent performance improvements under several label distribution shifts in small to mid-scale benchmarks and large-scale benchmarks, respectively. A possible follow-up question is the efficiency of DART-split in benchmarks like CIFAR-1OC-LIC which are not large-scale benchmarks. As summarized in Table 7 DART-split shows about 10% improvement under severe label distribution shifts (p = 100) in 1IFAR- and it performs comparably to naive BN Adapt at lower shifts (p 1, 10). While original DART outperforms DART-split on CIFAR-10C-LT for all ps, DART-split's success in significantly enhancing predictions under severe shifts while maintaining performance under milder conditions notable. On the other hand, in Table DART-split shows superior performance on largo-scale benchmarks compared to the original DART. Therefore, we recommend using the original DART for small to mid-scale benchmarks and DART-split. for large-scale benchmarks.
B.3 DART-split on Large-scale Benchmark
In Table 8 we present the experimental results on C0CAR-000C-imb and ImageNet- C-imb of several imbalance ratios. We can observe that DART-split achieves consistent performance improvement of 16.7-39.89. on CIFAR-100C-imb of imbalance ratios 200-50000, and 0.2-4.8% on ImageNet-C-inng of imbalance ratios 1000-500000, respectively, when combined with BNAdapt. Specifically, on CIFAR-100C-iml BNAdapt's performance significantly decreases as the imbalance ratio increases, falling well below that of NoAdapt. However, due to the effective prediction refinement by DART-split, it consistently outperforms NoAdapt by more than 4% across all imbalance ratios. DART-split can also be combined with existing TTA methods as reported in Table S We confirm that our proposed method consistently contributes to performance improvement in all scenarios. These experiments demonstrate that our proposed method effectively addresses the performance degradation issues of the classifiers using the BN layer, which were significantly impacted by label distribution shifts.
24

---

Published in Transactions on Machine Learning Research (02/2025)
Table 8: Average accuracy (%)
DART-split-applied TTA methods on CIEAR-100C-imb and ImageNet- ImageNet-C-immb
DIFAR-100C-iml
Image ImtgeNet-C-imm
IRI
IR200
IR500
IR50000
IRI
IR1000
IR5000
IR500000
NoAdapt
01.5:1..3
41.520.3
41.420.3
41.540.3
18.0=#.00
18.0
18.020.0
18.1=0.1
BdAdapt
59.810.5
29.110.5
18,810.4
9.310.11
31.50000
19.810.1
8.410.1
3.410.0
AaAdd---DARTT-pplII (ours)
59.7:10.5
45.810.2
50.210.3
49.110.77
31.5:0.0
20.0:10.2
11.51066
8.210.5
TENT
62.0+0.5
25.4+0.4
15.4+0.5
7.1+0.1
43.1++.22
18.7+0.2
4.3+0.1
1.2+0.0
DANT= (ours)
61.9+0.5
43.9+0.4
48.7+00.5
47.7+00.7
43.1+0.2
19.2+0.2
9.0+0.7
6.1+0.5
SAR
61.0:1..6
26.720.4
16,520.4
7.8.0.1
40.5:1.11
24.3:1011
9.6:10.1
3.540.00
SAR+DART-se (ours)
60.940.6
44.720..3
50.020.3
49.1
40.540.1
24.4+0.2
12.1+0.6
8.320.6
ODS
62.440.4
0..6
43.7:0.4
38.710.3
43.5:0.2
30.3+00.1
14.6+:.3
9.1+0.11
DDS+DART-spli (ours)
62.4+0.4
02.5
50.7+10.7
49.3+
43.5+10.2
30.3+0.2
17.1
13.3+0.6
C More Related Works
C.1 TTA method Utilizing Intermediate Time
Some recent works (Choi et al. 2022: Limn el al. 2022 Park al. 2023 try to prepare an unknown Lest-lime distribution shift by utilizing the training dataset at the time after the training phase and before the test time, called intermediate time. SWR (Choi et al.) 2022) and TTN (Lim et al., 2022) compute the importance of each layer in the trained model during intermediate time and prevent the important layers from siguificantly changing during test time. SWR and TTN compute the importance of each layer by computing cosine similarity between gradient vectors of training data and its augmented data. TTN additionally updates the importance with subsequent optimization using cross-entropy. Layers with lower importance are encouraged to change significantly during test time, while layers with higher importance are constr to change minimally. On the other hand, our method DART trains prediction refinement module during intermediate time by experiencing several batches with diverse class distributions and learning how to modify the predictions generated by pre-trained classifiers to mitigate the negative effects caused by the class distribution shift of each batch.
C.2 TTA methods considering Sample-wise Relationships
Some recent works (Boudiaf et al. 2022 Iwasawa & Matsuo 2021 Jang al. 2022 focus prediction modification using the nearest neighbor information based on the idea that nearest neighbors in the embedding space of the trained classifier share the same label. T3A (Twasawa & Matsuo 2021 replaces the last linear layer of the trained classifier with the prototypical classifier, which prodicts the label of test data to the nearest prototype representing each class in the embedding space. LAME (Boudiaf et al. 2022) modifies the prediction of test data by Laplacian-regularizeee maximum likelihood estimation considering clustering information.
C.3 TTA methods considering Class-wise Relationships
2021 Kang et al., 2023 Zhang al., 2023) focus on preserving class-wise wise relationships as domain-invartin information during test time. For instance, the method in (Kang et al. 2023) aims to minimize differences between the class- wise relationships of the test domain and the stored one from the training domain. CRS (Zhang et al. 2023) estimates the class-wisc relationships and embeds the source-domain class relstionship in costrastive learning. While these methods utilize class-wise relationships to prevent their deterioration during test-time adaptation, DART takes a different approach by focusing on dircctly modifying the predictions. DART considers class-wise confusion patterns refine predictions, effectively addressing performance degradation due to label distribution shifts, without explicitly enforcing preservation of class-wise wise relationships.
Some TTA methods (wwasawa & Matsuo
25

---

Published in Transactions on Machine Learning Research (02/2025)
C.4 Loss Correction Methods for Learning with Label Noise
In learning with label noise (LLN), it is assumed that there exists noise transition matrix T, which determines the label-flipping probability of sample from one class to other classes. For LLN, two main strategies have been widely used in estimating T: 1) using anchor points (Xia et al. 2019 Yao et al. 2020), which are defined as the training examples that belong to a particular class almost surely, and 2) using the clusterability of nearest neighbors of a training example belonging to the same true label class (Zhu et al.j 2021). LLN uses the empirical pseudo-label distribution of the anchor points or nearest neighbors to estimate T.
For TTA, on the other hand, the misclassification occurs not based on fixed label-flipping pattern, but from the combination of covariate shift and label distribution shift. To adjust the pre-trained model against the covariate shifts, most TTA methods apply the BN adaptation, which updates the Batch Norm statistics using the test batches. However, when there exists a label distribution shift in addition to the covariate shift, since the updated BN statistics follow the test label distribution, induces bias in the classicr (by pulling the decision boundary closer to the head classes and pushing the boundary farther from the tail classes as in Appendix 2. Thus, the resulting class-wise wise confusion pattern depends not only the class-wise wise relationship in the embedding space but also on the classifier bias originated from the label distribution shift and the updated BN statistics. Such classifier bias has not been problem for LLN, where we do not modify the BN statistics of the classifier at the test. time.
Our proposed method, DART, focuses on this new class-wise confusion pattern and is built upon the idea that if the module experiences various batches with diverse class distributions before the test time, it can develop the ability to refine inaccurate predictions resulting from label distribution shifts.
D Theorectical Analysis
D.1 Motivating Toy Example Confusion Patterns Reflecting the Class Relationship by BNAdapt
Table 9: Notation and definitions used in Section D.1
Symbol
Meaning
N(u, 12)
Multivariate Gaussian distribution with mean μ and variance o²
P4
Mean vector of class (i 1,2,3,44 Variance shared across all classes
0²
d
Constants controlling class separation (d> 1, 1)
A
Test-time distribution shift vector
A(-)
Bayes classifier for prediction Mean-centered Bayes classifier for prediction
R(-)
(v)
Class prior probability during training (uniform: 1/4)
Class prior probability during testing (imbalanced distribution)
14
Shifted mean of class after lest-Lime mean centering
2(-)
Cumulative distribution function (CDF) of the standard normal distribution
To understand the effects of test-time distribution shifts, consider Bayes classifier for four-class Gaussian mixture distribution with mean centering, which mimics batch normalization. Let the distribution of class be N(u,, 0² at training time for = 1,2,3 3, and 4, where μ4 E IR² is the mean of each class distribution. We set the mean of each class as μ4 = (d, 3d), #2 = (-d, 3d), μs = (d, -Bd), and #4 = (-d,- -Bd), where B controls the distances between the classes, and we assume that 3 > Moreover, we assume that the four classes have the same prior probability at training time, i.e., Pu(y = i) 1/4, 1, and 4. Since the class priors for the training data. equal, the Bayes classifier h predicts x to the class when
Pur = (zly j), iti
(9)
26

---

Published in Transactions on Machine Learning Research (02/2025)
due to Bayes' rule. Then, we have
1,
if 0,32 0:
2,
if 1 < 0,02 0;
((a))
(10)
3,
[1 > 02 0:
4, if 21 < 0, 22 0.
Initially, we assume that the input data. distribution is shifted by A IR2 at test time studied in prior works such as Stojanov et al. (2021): Yi et al. (2023). Specifically, the distribution of class is modeled N(H; + o2 I2) for i = 1,2,3, and 4 at the test time. Employing mean centering moves the distribution of class from W(4 + A, 12) to ((4, 12). Consequently, we demonstrate that the mean centering effectively mitigates the test-time input data distribution shift.
Furthermore, we consider a scenario where the class distribution also shifted along with the input data distribution. We assume that the class distribution is imbalanced, similar the long-tailes distribution,
(y 1) p,
(11)
(y 2) = 1/4,
(12)
Pro = 3) =1/4,
(13)
Ptoly = 4) == 1122 112-10
(14)
Assume that 1/4<
< < 1/2. Due to the mean centering, the distribution of class i is shifted to N(4, o 2²)))
where 1e is the shifted class mcan as follows:
1 = (3322 2p) (8/2--22) 2p)
(15)
μ1 = ((-1/2 - (2). - 2d) ng = ((3/2 - 2p),, (-1/2 2p) 1/ = ((-1/2 -p) 2d)
(16)
(17)
(18)
Then, the probability that the samples from class are wrongly classified to class 2 can be computed as
Pr s)
(NNorm()))) 2
Pr
[Mxx) = 2]
(19)
Pr
[2] <0,00 > 0]
(20)
==(0),02)~ (1/ 22)
(3/2 2p)d
{1- $ (- (33/2 2p)2-pp)B80
(21)
o
where Norm is a mean centering function and & is the standard normal cumulative density function. Similarly, the probability that the samples from class 2 are wrongly classified to class can be computed as
Pr s>N(uutA (h(Norm(2)) =1] =]]= {1 =1]= + (1(11/2-2))) 2p)d
{1 Φ (- (3/2-2p))50 -p) }
(22)
σ
σ
1], where h(.) = h(Norm()) )) is modeled BN-adapted classifier. With similar computations, we can obtain Prz-Nirr+A [xxx) i
Since 1/4 p 1/2, we have Pr [T()) 2] > Pr.-NWu=+A.
1]. Vi 2,3, and In other words, the proba- bility that the samples from the class of larger number of samples are confused the rest of the classes is greater than the inverse direction.
27

---

Published in Transactions on Machine Learning Research (02/2025)
Number of samples
Few
Many
Class
Class 1
Class 2
Class
Class
Class
malhit M0:0 a]
[800
3]
Input data label
Class
Class 3
Mean centering
Class
Class
distribution shifts
Class
Class
Figure 8: Illustration of the 4-class Gaussian mixture model from Section D.1 The two lines indicate the Bayes classifier decision boundarics, partitioning the feature space into regions corresponding different classes. The patterned areas highlight regions where misclassification occur. Specifically, the red patterned area denotes regions where samples from Class 1 are misclassified as Class 2, while the green patterned area represents regions where samples from Class are misclassified Class 3. Two notable confusion patterns can be observed when both input data and label distribution shifts occur, and mean centering is applied. First, Class 1, which has a larger sample size, is more prone to misclassífication into other classes compared to smaller classcs. Second, misclassffiations are more likely to occur toward close classes in the fcature space. For instance, Class more frequently confused with the closer class (Class 2) than with other classes.
The probability that the samples from class are wrongly classified by h as class 2,3, and can be calculatod follows:
(3/2 - (3/2-2p)) 2p)d
(3/2 2p)
Pr a-N(nn
[h(x) = 2] = +b
(23)
0
(3/2 - 2p)
/_33/2 2p)dd
Pr DNN(mi+A
[1x))) = 3] = +b
(24)
0
(3/2 - 2p)
(3/2 2p)d
Pr [h(x)) = 4| = +
(25)
(=N(HL+A,0011)
Note that $ (- (-13/2-2p/)) 2p) has the following properties: Since 1/4 < p 1/2, $
(313/2 2p)
<1 1/22 Since
8 ① (- (11122 (3/2-2p) op) + (3/2-2p)a 2p)d
öp ( (3/2--20))8 B exp
(3/2 (3/2-20) 2 8
where Cj is a
a positive constant, independent of and 8, which decreases as grows for 3 Thus, say that.
2
(3/2-2p) 2pjd
(1) The probability that the samples from the head class (class 1) are confused to tail classes greater than the reverse direction, specifically, Pr (f(()) = Pfg=:NattA (h(x)) = 1], Vi A 1, where h(()) == h( h(Norm(-) is composition of mean-centering function and Bayes classifier obtained using training dataset, modeling the BN-adapted classifier.
(2) The probability that a sample from the head class is confused to the closer class larger than that to the farther classes. Specifically, Pi 2] > Pln-N(u [h(x) 3] (M) = 4].
(3) As the class distribution imbalance p > 1/4 increascs, the rate at which misclassification wards spatially closer classes increases is greater than towards more distant classes. Specifically,
Op "PawWit+Aoo [b(x) oe PionW(uts (T(x) = when 2a < d.
28

---

Published in Transactions on Machine Learning Research (02/2025)
The illustration of the class-wise confusion pattern caused by test-time imput data and label distribution shift in 4-class Gaussian mixture model can be found in Figure g The effects of test-time label distribution shift can be consistently observed not only in this toy example but also in real dataset experiments, e.g., CIFAR-1OC-LT (Section
D.2 Meaning of DART's Logit Refinement Scheme
DART outputs a square matrix W of size K and vector b of size K to correct the inaccurate BN-adapted predictions caused by the test time distribution shifts through an affine transformation of the classifier output (logit). Since the affine transformation can be interpreted as a simple matrix multiplication, achieved by modifying the input vector by adding 1 and including the vector into the square matrix, we focus on analyzing the theoretical meaning of the square matrix W generated by DART. To understand the theoretical meaning of W obtained by DART's training, has a similar elfect as the batch normalization as we discussed in Section D.I As demonstrated in Section D.11 the mean centering effectively mitigates a test-time input data distribution shift. Thus, we focus only on the label distribution shifts in the following analysis. Let the distribution of class be W(s,, oTa). Generally, the classifier output (logit) is computed as the product of features and classifier weights. With the mean centering, the logit for an example x for class i can be represented by the product of mean-centered features and centeren class centroids, i.e.,
consider K-class classification problem with mean centering, which
1(x); = (
PaHl)( Σ pbPp) F Vi [K] aE(K]
E
(26)
bE(K]
where p = (pi, .2...., pk] BIxk for can be computed as
K is the label distribution of training dataset. In the matrix form, the logit
(x) (I - pp) 1 1xxIp)) = (a- (x-P))) (p))p((Tx 1xx1p)
(27)
where 1xx1 is matrix of size K x 1 for which all the elements are 1, and μ = lux, .2..... "K] E R is the concatenated class centroids. We assume that the logits are robust to the changes in the class distribution of the training dataset, i.e.,
((x) = (x - (--pp)p [LK- - 1KxIp)) = (x - rT)) (Ik LxxIr)",
(28)
when r [r1, T2...... rk] satisfies r; 2 and E.
At the test time, when the label distribution is shifted from p [p1, P2..... mean-centered logit for an example after the shift can be computed as
... Pkl to q [91, 121:92,-.. ................................ .Kk,, the
((x) = (c qr)) (Tx - [xxp))
(29)
since only the features are newly mean-centered while the classifier weights are unchanged similar to BN Adapt (Nado al.) 2020 Schneider al.] 2020).
DART learns the square matrix W E R to refine the BN-adapted classifier's output to the original one by multiplying it with W. Specifically, we obtain W which minimizes the below optimization problem when we use the L2 norm,
W = arg min ExeRIxa [[(2 - pp)p (Ix - 1xxIP) - (x - -u) (IK (xxxp))T[l
(30)
W
s arg min Xx [1[x - qu)pu [[ke - -xxig)T - (x pp [Jx 1xxIp) 1xx1p)7W1B3
(31)
W
arg min Ex [l(x gu) (Ix -KxIq)"--- 1xxx1)
(32)
W
Since we match these two logits for any x, we can rewrite the above equation as follows:
W arg min 1| (Ix 1xxIq)= 1ht (Tx T
(33)
W
29

---

Published in Transactions on Machine Learning Research (02/2025)
Since d is greater than K generally, the least-square solution for the above equation is
W [[Ix 1x 1xxIp)pp" (Ix - IRxIP) (Ix 1xx19)*
(34)
with an assumption that (IK 1xx 1xiIp)mp" (IK - 1 is invertible. Then, we can observe that W is determined by only three different components: the label distributions of training and test datasets (p and q, resp.) and the class-wiss wise relationship, i.e. the relationship among the class centroids (up" e RK**). In find that W is related to the relationships among class centroids, not the exact locations of the class centroids.
particular,
In contrast to the re-weighting methods [Garg et al., 2023 Hong et al. 2021 Menon et al. 2020), which do not take into account class relationships for prediction refinement, DART considers class relationships for prediction refinement as evidenced by the above W*. Thus, DART can address the errors with confusion patterns arising from test-time distribution shifts, in which the label distribution is also shifted.
E Confusion Matrices of BN-adapted Classifiers on CIFAR-10C-LT
In Figure [ we present confusion matrices of the BN-adapted classifiers for three different corruption types (Gaussian noise, defocus blur, snow) from each of the three corruption categories (noise, blur, and weather) among 15 pre-defined corruptions. In Figure 9 we present confusion matrices for all 15 corruptions including clean CIFAR-10 test dataset with various levels of label distribution shifts (o 1, 10, 100), respectively. While test accuracy may not perfectly be matched across the corruptions, we can observe that (1) the accuracy in head classes (with smaller class index) significantly decreased and (2) confusion patterns tend reveal increasingly pronounced class-wise confusion patterns as the imbalance ratio p rises from to 100. Based on these observations, we conjecture that the prediction refinement module trained only on the training data the intermediate time remains consistently effective in modifying predictions for the test da datasets.
to be consistent across different corruptions as we described in Section 2 Additionally, Figure
30

---

Published in Transactions on Machine Learning Research (02/2025)
139
139
C34
XMC
10-0000
50-1000
313
200
4--000
519
75-200
Figure 9: Confusion matrices of BN-adapted classifiers CIFAR-1OC-LI with p 1. We mark the cases where the confusion rate exceeds 11% with red squares.
31

---

Published in Transactions on Machine Learning Research (02/2025)
2000
Se--211
300 LK
530
30
125
ANC
DNL COMTAN
0.00
324
0.00 3-1
UK
2.0
24--001 300
542
Confusion matrices of BN-adapted classifiers CIFAR-1OC-LT with p 10. We mark the cases where the confusion rate exceeds 11% with red squares. We can observe notable accuracy decreases in classes with large amounts of data, and similar confusing patterns regardless of the corruption types. Compared to = 1, we can observe more pronounced wlasswwise confusion patterns.
Figure 10:
32

---

Published in Transactions on Machine Learning Research (02/2025)
1000
550
D20 BMC
120 100
Ch-
DEL COMTANA
20-2211-
0.00
0.00
300
UK
500
2.0 4.0
20-220 300
50
Figure 11: Confusion matrices of BN-adapted classifiers on CIFAR-1OC-LT with p 100. We mark the cases where the confusion rate exceeds 11% with red squares. We can observe notable accuracy decreases in classes with large amounts of data, and similar confusing patterns regardless of the corruption types under the class distribution shift. Compared to p = 1, 10, we can observe clearer class-wise confusion patterns.
33

---

Published in Transactions on Machine Learning Research (02/2025)
F Prediction Refinement Module Outputs on CIFAR-1OC-LT
(a) p=1
(b) p= p= 10
(c) 100
Figure 12: Prediction refinement module outputs on CIFAR-IOC-LT of 10, 100 without regularization
(f.c., a =
(a) p=1
(b) = 10
(c) = 100
Figure 13: Prediction refinement module outputs on CIFAR-1OC-LT of p= 1, 10, 100 when a = 0.1
(a) p=1
(b) p= p= 10
(c) p 100
Figure 14: Prediction refinement module outputs on CIFAR-1OC-LT of p= 1, 10, 100 when a =
In Figurcs 12
present the W and b, generated by 9g when inputting the averaged pscudo label distribution and prediction deviation computed on the whole test data for CIFAR-10C-LT under Gaussian noise with different a values (0, 0.1, and 1) in equation 3 The (a), (b), and (c) of each figure show the outputs of Is when class imbalance p is set to 1, 10, and 100, respectively. In Figures [2[ we commonly observe that both W and assign high values for the head classes (classes 0-1). This prediction refinement of DART can correct the predictions instances from the head classes that are misclassified other classes, recovering the reduced test accuracy of the head classes (shown in Figure D. As shown Figure 15 of Appendix G, this prediction refinements scheme of DART can modify the averaged pseudo label distribution to be closely matched with the ground truth label distribution, which is originally significantly disagreed.
we can observe the impact of the regularization of equation [ which encourages Jø lo output an identity matrix and a zero vector for nearly class-balanced batches. Specifically, the values of the off- diagonal of W and become almost for p= 1. The classifier logit of cach data point reflects not only the class which the data point belongs but also the characteristies of the individual data point (Hinton al.) 2015).. For example, even for data points in the same class, the classifier logit can vary depending on factors such background color, thereby containing information
Morcover, by comparing Figure 12 and Figure 13
14
34

---

Published in Transactions on Machine Learning Research (02/2025)
specific to each data point. Therefore, in the absence of test-time class distribution shift, where there is class-wise confusion pattern to reverse, the classifier logit modified by the prediction refinement poses a risk of damaging the information of each data point, even if the prediction itself remains unchanged. By adding the rogularization, we can prevent the unintended logit refinements for the nearly class- balancod batches since the batches might have no common class-wa wise confusion patterns originated from label distribution shifts (Figure D1.
We experimentally demonstrate the effect of regularization (equation and present the results in Table 10 Without regularization (a 0), apPT-applied TENT shows approximately 2% lower performance under label distribution shift (p 1) compared to other a values. It indicates that DART without regularization makes unnecessary adjustments, harming the subsequent test-time adaptation when there is no class distribu- tion shift. On the other hand, the TTA performance remains stable across all ps when a is set between 0.01 and 1. This result demonstrates the effectiveness of regularization and the robustness DART changes a.
Table 10: Accuracy (%) of DART-apppied BNAdapt and TENT with different a values CIFAR-1OC-LT.
CIFAR-1OC-LT
Method
p
p 10
P 100
85.3
85.5
85.8
0.01
85.4
85.1
85.7
BNAdapt DART
0.1 (used)
85.2
84.7
85.1
0.5
85.3
83.9
84.8
85.3
83.5
84.7
84.4
86.2
88.3
0.01
86.1
86.9
88.7
TENT + DART
0.1 (used)
86.4
86.8
88.2
0.5
86.6
86.5
87.9
86.5
86.2
87.8
G More Experiments
Table 11: Average accuracy (%) on CIFAR-IOC-LT using condensed training data. DART with condensed training data achieves similar performance gains as with original training data, improving BN- adapted classifiers under label distribution shifts without any degradation performance when there no label distribution shift.
Method
Avail. data at int. time
p=
p = 10
= 100
BNAdapt
85.2+0..0
79.0=0.1
67.0
Training data
85.2+0.1
84.7=0.1
85.1+0.3 81.410.4
DAARR
Condensed tr. data
85.240.0
82.4=0.3
G.1 Reducing the Dependency of Labeled Training Data in the Intermediate Time Training of DART
As described in Section 5 and Appendix C some recent TTA methods utilize the training data to prepare adaptations to unknown Lest-time distribution shifts during the intermediate time. DART also uses the training data to construct batches with various label distributions to learn how to detect label distribution shifts and correct the degraded prodictions caused by the shifts. Under a strict situation in which training data is unavailable due privacy issues, we can use condensed training data, which small synthetic data that preserves essential classification features from the original training data while discarding private information.
35

---

Published in Transactions on Machine Learning Research (02/2025)
It could prevent privacy leakage (Kng et al. 2023 We report experimental results CIFAR-1OC-LT for the case when using the condensed training data instead of the original one in Table 11 We use the publicly released condensed dataset of CIFAR-10 by Cazenavette et al. (2022) consisting of only 50 images for each class. Even when using the condensed data, DART achieves similar performance gains as when trained with the original data. Specifically, DART improves the degraded performance caused by label distribution shifts (+3.4/144) 43.4% for p p= 10/100) without any degradation in performance when there no label distribution shift (p = 1). This demonstrates that DART training is possible without the original training data.
Table 12: Average accuracy (%) on mixed CIFAR-10C. Even in the mixed CIFAR-10C dataset, DART effectively mitigates performance degradation caused by label distribution shifts resulting from Dirichlet distribution sampling.
Method
No label dist. shifts
Under label dist. shifts
NoAdapt
71.7=+0.0
71.7++0.0
BNAdapt
75.7++..1
20.4++0.1
75.8+0.11
81.5+0..8
ROID Marsden et al. 2024) ROTD DART
80.
12.9#1.2
80.4+0.1
63.8+2..
G.2 DART in Mixed Domain TTA Settings
To verify DART's effectiveness in a complex scenario, we consider a 'mixed domain' setting. Here, corrupted test data from 15 types of corruptions are combined into a single test data, as in Marsden et al. (2024). Under this mixed domain setting, we consider both no label distribution shifts and label distribution shifts, where for label distribution shifts, the test data is non-i.id sampled using Dirichlet sampling with 0.1. We summarize the results for the original v.s. DART-appliod TTA methods in Table [2] The results indicate that 1) DART effectively addresses the performance degradation of the BN-adapted classifier even in the mixed domain setup (+61.1%), and 2) while ROID (Marsden et al.; 2024). state-of-the-art method in mixed domains, shows a significant performance drop of 67.4% under label distribution shifts, DART leads to an improvement of 50.9% in performance of ROID. DART's prediction refinement could be effective in mixed domains since is based on the finding that the BN-adapted classifier's predictions exhibit consistent class-wist wise confusion patterns due label distribution shifts, regardless of corruption type (Fig. D We expect that DART's prediction refinement will also be elfective in other challenging sit situioions where both covariate and label distribution shifts occur.
G.3 Robustness to Changes in Test Batch Size
Table 13: Experiments balanced CIFAR-10C with different batch sizes.
Test batch size
Method
4
8
16
32
64
200
NoAdapt
71.7=00.0
71.7=00.0
71.7+0.0
71.7+0.0
71.7+0.00
71.7=1..0
BNAdapt
71.540.3
76.740.1
81.110.1
83.410.1
84.5++..0
85.210.0
BNAdapt DDART (ours)
74.3+0.1
80.4+0.1
82.0+0.1
83.5+0.1
84.6+0.1
85.2+0.1
TENT
19.7+0.6
37.611.2
60.211.2
75.7+0.5
86.310.1 83.6
82.510.3
TENT+DART (ours)
51.342.3
68.141.0
75.6+0.4
80.2+0.5
86.5+0.2
In Table 13, we evaluate the effectiveness of DART under small test batch sizes on balanced CIFAR-10C. Even if the label distribution of the test dataset remains balanced, when the test batch size becomes small,
36

---

Published in Transactions on Machine Learning Research (02/2025)
class imbalance naturally occurs within the test batch as discussed in SAR Niu et al. 2023 As the test batch size gets smaller from 200 to 4, the performance of BNAdapt gets worse. For example, the performance of BNAdapt is slightly worse than the one of NoAdapt when the test batch size is set to 4. However, DART successfully improves the degraded performance of BNAdapt by about 3% under a small test batch size (c.g. or 8).
G.4 Experimental Results using Different Hyperparameters CIFAR-10C-LT
Table 14: Experimental results using different combinations of the hyperparameters CIFAR-IOC-LT
hidden dinension of (ased)
intermedtate batch
8100
BNAdlps
65,260.0
67.060.1
67.0g001
85226.0
790
07.046.1
86.240.0
79:040.1
67.040.1
32
BNAdapo+ DART (ouns]
(uard)
65.34:000
84.35005
$5.2201
$4.7202
848-02
8322001
847=0.1
85.130.3
80.360.00
84.6002
80.12004
138
85,4+00
864+0.1
80,5403
$5,4+0.0
815+00
51.0-03
83/1+6.0
847-0.1
81.9-0.2
85.1+0.0
84.94611
85.3+0.4
In Table report the experimental results when the intermediate batch size and the hidden dimension of Yo are set to 64 and 1,000, respectively. In Table [4] we summarize the experimental results using different combinations of the hyperparameters on CIFAR-1OC-LE We can observe that the performance gain of DART is robust to changes in the hyperparameters for most cases. We observe that the performance improvement is about 14% when the intermediate batch size is set to 128 for CIFAR-10C of p= 100, while shows about 18% when the intermediate batch size is set 32 or 64. This because the prediction refinement. module 9o does not experience sufficiently severe class imbalance under the same Dirichlet distribution when the intermediate batch size increases. For example, when there two batches of the same size having class distributions [0.9, 0.1] and [0.1, 0.9), the combined batch has class distribution of [0.5, 0.5]. Therefore, this can be addressed by modifying the Dirichlet distribution to experience severe class imbalance by reducing 5 or Ndir.
Table 15: Performance of DART-applied BNAdapt CIPAR-10C-LT with different intermediate-time training epochs
CIFAR-IOC-LI'
# of epochs
p 10
p= 100
82.0
80.6
75.7
85.3
84.5
81.7
10
85.4
84.9
83.3
25
85.3
84.8
84.8
50 (used)
85.2
84.7
85.1
We summarize the experimental results on the effect of (raining epochs for the prediction refinement module 9o during the intermediate-time training Table 15 We set the training epochs for intermediate time training for CIFAR-10. To assess the convergence of 9o training, we conduct experiments by roducing the number of training epochs from to 25. Notably, training for only 10 epochs achieves comparable results to 50 epochs, with just 1.8% gap observed for CIFAR-1OC-LT with p 100. This demonstrates the efficiency of DART's intermediate-iine time training, significantly enhancing its scalability for practical use.
G.5 DART on CIFAR-10C-imb
We summarize the experimental results on CIFAR-10C under online imbalance setup in Table 16 We observe that the DART-applied TTA methods consistently show improved performance than naive TTA methods as shown in the experiments on CIFAR-1OC-LT of Table I BNAdapt's performance docreases as the imbalance ratio increases, and it shows worse performance than NoAdapt when the imbalance ratios are set 20-5000. However, DART-applied BNAdapt consistently outperforms NoAdapt across all imbalance ratios.
37

---

Published in Transactions on Machine Learning Research (02/2025)
Table 16: Average accuracy (%) of DART-applied TTA methods on CIFAR-10C-imb
CIEAR-10c-imb
IR.
IR5
IR20
IR50
IR5000
NoAdapt
71.610.1
71.7#00.1
71.7=0.1
71.610.1
71.7+0.1
BNAdapt
85.3.1.11
77.1+0.3
50.0=0.4
34.610.4
20.3
D (ours)
85.3+:..2
83.340.1
76.2
78.210.1
82.4
TENT
86.5+0.2
75.840.6
47.3+0.1
32.6+0.4
19.1+0.1
TENT+DART (ours)
86.510.3
83.240.1
73.6=0.3
71.440.6
71.5+11
Table 17: Average accuracy (%) on DomainNeet-126. X2Y refers to a scenario where pre-trained model, trained in domain X, is tested in domain Y. We can obscrve that DART achieved consistent performance improvements on DomainNet-126.
Method
12c
x2p
12s
c2r
c2p
c28
p2r
p2c
p2s
62r
82c
62p
avg
BN Adapt
52.65
61.57
46.38
63.12
48.68
47.93
73.54
52.67
50.96
66.32
57.86
57.56
56.60
BN BNAdaptDDART
53.40
62.20
46.95
66.07
51.74
50.21
75.05
54.49
52.00
69.73
60.73
59.82
58.53
(+1.93%)
G.6 DART on DomainNet-122
We test the effectiveness of DART on DomainNee-116,, and the experimental results are summarized in Table 17 We follow the experimental setting described in Marsden et al. (2024) and set the test batch size to 32. We can observe that DART achieved consistent performance improvements of 1.93% on average on Nom We would like to emphasize that DART consistently achieves performance gains large-scale benchmarks such OfficeHome and DomainNet. In the large-scale benchmarks, the batch size is generally significantly smaller than the nunber of classes. Then, even if the class distribution of the test dataset differs greatly from that of the training dataset, the change in class distribution within each test batch would be minimal. In such cases, the performance improvement due to DART seems to be less significant compared to small-scale datascts like CIFAR-10C-LT in Table 1, but DART is still effectively adjusting inaccurale predictions by label distribution shifts.
Table 18: Average accuracy (%) of MEMO on CIFAR-1OC-LT. MEMO (stored BN) performs worse than NoAdapt while it does not show a performance decline under label distribution shifts. On the other hand, MEMO (adapted BN) exhibits performance degradation due to label distribution shifts, similar to BNAdapt. DART can effectively address these performance degradations.
Method
==
p == 10
== 100
NoAdapt
71.7+0.0
71.3+0.1
71.4+0.2
MEMO
(stored BN)
67.610.1
68.910.3
71.4+0.1
MEMO
(adapted BN)
87.0
85.0+0.2
73.6+0.1
MEMO
(adapted BN) + DART
86.8+0.3
87.8+0.1
87.1+0.4
G.7 Applying DART to MEMO
We test the effectivencss of MEMO on CIFAR-10C-LT and try to combine MEMO and DART. MEMO finc- tunes classifiers to minimize the marginal entropy of the classifier outputs across different data augmentations. We consider two variants of MEMÓ introduced in (Goyal et al. 2022). The first variant, MEMO (stored BN), uses stored BN statistics and resets the classifier for each test batch. The second variant, MEMO (adapted BN), uses the BN statistics computed on each test batch and continuously adapts the classifier like TENT. In Table 18
observe that MEMO (stored BN) does not show performance decline under
38

---

Published in Transactions on Machine Learning Research (02/2025)
0.40
gt label dist (p-1)
0.40
gt labal dist [g=1]
avg pl dist (p-1)
avg refined (p-1)
0.35
label dist (p=10)
label dist (c=
0.30
evig pl dist (p=10)
0.30
avg refined pl dist (p=10)
dctribution
gi label dist (p-100)
Class distribution
gt label dist (g-
0.25
aist (p-100)
retined (p-100)
0.20
0.20
0.15
0.10
0.10
0.05
0.00
0.00
Class
Class Index
Figure 15: Visualizations of ground truth and averaged pseudo label distributions of (left) ANAd and (right) DART-applied BN Adapt on CIFAR-1OC-LT The averaged refined pscudo label distribution (solid line) generated by the DART-applied BNAdapt closely matches the ground truth label distribution (dashed line), but the pseudo label distribution of BNAdapt does not match.
label distribution shifts on CIFAR-10C as shown in Zhao et al. (202)), but performs worse than NoAdapt. On the other hand, MEMO (adapted BN) exhibits performance degradation due label distribution shifts, similar to BNAdapt (-2/13.4%) for p = (0/100). However, we can observe that DART can effectively address these performance degradations. Through the above experiments, explored the potential for combining DART and MEMO (adapted BN).
G.8 Comparison of Averaged Pseudo Label Distribution of BNAdapt and DART-applied BNAdapt
In Figure 15 we present. the ground truth label distribution and the averaged pseudo label distributions generated by (Ieft) the BN-adapted classifier and (right) the DART-applied BN-adapted classifier for CIFAR- 10C-LT with different ps. In Figure 15 (left), distribution of the BN-adapted classifier does not perfectly match the ground truth label distribution, it still contains information about the direction and severity of the label distribution shift. However, except when is 1 (i.e., balanced), the averaged pseudo label distribution differs significantly from the ground truth label distribution. Specifically, the averaged pseudo-label distribution slightly closer to uniform compared to the ground truth label distribution. On the other hand, Figure 15 (right), we observe that the averaged psoudo label distribution generated by the DART-applied BN-adapted classifier almost perfectly matches the ground truth label distribution. Through these experiments, can see that the prediction refinement by DART can accurately estimate the label distribution, which is advantageous for subsequent test-time adaptation.
can observe that although the averaged pseudo-label
Transition Matrix Estimation by Noisy Label Learning Method
HOC (Zuu et al. 2021 estimates the noisy label transition matrix T, which represents the probabilities of clean labels being flipped noisy labels, for a given noisy label dataset under the intuition that the nearest. neighbor in the embedding space of trained classifier fe shares the same ground truth label. In the case of noisy labels caused by random transitions, the noisy transition matrix can be predicted using feature clusterability nearest neighbor information in the embedding space. However, shown in Figure 16 (a) left, the nearest neighbors in the embedding space already have the same predictions of BN-adapted classifiers, making it difficult for HOC to estimate the noisy transition matrix using feature clusterability (Figure 16 (a) right). Thus, the estimated T by HOC similar to the identity matrix (Figure 16 (b)). In this paper, we thcoretically and experimentally analyze the misclassification patterns and propose a TTA method DART address performance degradation by learning the prediction refinement module which modifies the inaccurate BN AdAdppt predictions due to label distribution shifts.
39

---

Published in Transactions on Machine Learning Research (02/2025)
Gooand with label
doo endiovatdd
0.90
600
OCB
8:00
0C3
0.9
8000
0:90
031
533
0.01
081
(a) T-SNE plots for BIFAR-10C-TE with Gausslan noise, p=100
(b) Estimated T by HOC when p 100
Figure 16: (a) T-SNE plots of test data with ground truth labels (left) and their predictions (right) for CIFAR-10C-LT with Gaussian noise, p=100 (b) Estimated T by HOC for CIFAR-10C-LT with Gaussian noise, p=100
Expansion to Vision Transformer (ViT)
be applied to the adaptation of classifiers using batch normalization (BN), we believe that DART is applicable to Vision Transformer (ViT) if it uses BN well. ViTs typically Layer Normalization (LN) instead of BN. However, since BN generally leads to faster convergence and has shown strong performance
Since DART
in vision tasks, there have been attempts to replace LN with BN in ViIs (Chen et al.) 2021 For instance, it was shown that replacing LN with BN in ViTs for the ImageNet classification task resulted a 1% increase in classification performance, evaluated by linear probing (Chen et al.) 2021). For the ViIs replacing LN with BN, we expect that applying DART will lead to significant improvements in the classification performance of the classifiers under label distribution shifts.
40

---

Published in Transactions on Machine Learning Research (02/2025)
J Full Results
Table 19: Average accuracy (%) on CIFAR-1OC-LT (p = 1)
Nukdapt
06412.
111110
27:120
33
10.2111
08,333
00.22113
1122301
DN (sees
033135
61 6:3-03
06:149
50--01
069-0
58.
TENT
3:3
535-00
063-11
131433
PI.
003335
033403
062-01
580-0
013333
063-11
081-013
060-13
196407
300
NOTE--AARZ
76221.
LAVE
263-35
9-3-1
195-01
195401
9034
DELIA
012632
127284
862613
185001
183011
574336
5123144
033232
013835
026-07
X0SH1
0--01
00000
41665
19560
385-13
DDR-DART
125287
13X301
533013
831113
EAR
005435
135-63
516003
200-31
880-03
005-07
416167
83511
(wees)
415-63
Table 20: Average accuracy (%) on CIFAR-IOC-LT (p- 10)
Matiad
Evanbe, Jobr
stat. poll
sxpoii__ 1000
ddona Ner
Eio Mer
DAEK Ni
Brott
34g
Catt Instaiona
JHS pesprooke
16.1on3
55.244.3
6464
57.38833
764518
553283
77.323
8432611
74.22.
(rm)
TEXT
T76.23
80.3055
572486
EEN/ ADAR (1mss
40321:1
223383
LAAK Tean:
404231
102005
474214
N012
T24213
162217
41421
03 117134
LAMI
TAXE-DART (1408)
8.1.07
356665
8-2H11
513252
761213
BRIEN-DART (eaes)
411-14
$15651
6:1465
R1386
TOGML
667660
DES
30005
71243-
20011
23151
123666
16 117880
SAR
N.1-13
T74083
152617
073603
655-11
573465
79 14
656600
77.46.11
PAR--DRRU (1nx))
837303
313395
5.7103
125133
967218
313184
S.3203
50
Table 21: Average accuracy (%) on CIFAR-TOC-LT (p = 100)
Melaal
sbytt B0de
supelie
plms M4e
Fie
Codic tresdenn
Ittilate
1p52 ovegroste
00.5411
135-01
541494
73 1464
050004
002431
215-03
165-0
083-13
010-05
75-440
65 155001
11107
437-13
095-0
880-11
005-04
TEST- (sem)
$6121D
192315
5632215
ST5294
5522342
000-11
NOTE
56.2221
136237
81A:11
756115
NOVTR-TART (wed)
26-11
51543
88-01
06041
LAME- DAII
SU112
12501
883214
156231
ThALL
705416
211-11
0-4-21
11:-13
13-01
(63451
(80-11
DDS
962211
15120
114221
365213
176213
175300
524887
TUALL1
DOR- (0100
71381.
134220
10X120
801213
92
57500
833:11
S18220
501311
516314
553111
832211
(82-04
FAR-DART
067-01
Table 22: Average accuracy (%) on PACS. X2Y refers to a scenario where a pre-trained model, trained in
domain X, is tested in domain Y. For example, a2c refers to a scenario where a pre-trained model, trained in the art domain, is tested in the cartoon domain.
Method
x2c
a2p
624
<2p
<24
p2n
1:2r
p2a
62c
+2p
Rgg
NoAdapt
66.0+0.0
97. 0.00
57.3-00
75:6-0.
90,2+0.0
72.2+0.0
73.2+00.
39.7+0.0
43.9+000
23.5+0.0
50.3-0.0
38.0-0.0
60.7+0.0
BNAdapt
76.61033
0...2..
70.0-02
03.
05.1:0.4
75.510.7
77.810.2
65.010.4
16.910.2
50.410.5
60.01..4
87.8.0.2
72.5.0.0
Mddpt--ARTT (oums)
76.0+0.3
96 8-0.
72 1-0.
84.4-06
354-002
74.7+0.5
76.1+0.3
62.0+0.4
65.0+0.7
74.1+0.2
76.7-0.2
74.0-066
76.4+0.1
TENT
78.610.9
97.8 8...55
17.
03.
97.1
7..1
81.7.0..
75.511.0
59.2125
60.111.1
710010
168116
76.810.6
MERT+DART.CN (owss
808888
97.7200
7922
889202
79.22:22
79.6
75.821.7
00.821.
S13206
77.620.7
71.020.5
81.620.4
PL.
77.6+0.4
96:9-0.6
70.-0.3
83.8-12
96A4-0.6
68.9+1.5
78.0+0.5
66.3+0.88
46.9+022
50.0+0.4
6866-14
56.8-0.2
72.4+0.2
PL4DART (ours)
78.01.0.8
96 8.0.2
71.7-008
85
05.010.4
7..9115
70.510.3
65.511.1
55.0110.7
71.11002
76.910.6
74.0226
76.710.4
78.2208
977 210.4
74
16.0
95.61.04
11.3114
11.411.0
72.5128
08.5.1.
7081128
70.1=22
09:2:0.6
79.1
N (ours)
80.7+1.0
97. 171.8
77.9=1.9
96.8-0.6
70.0=110 10
70.7+1.6
70.8+119
7166+111
78-++1.3
76.2=0.3
60.3-0.3
79.9=0.3
LAME
78.6::.44
96 0::0.1
84. 0:555
05.0:00.
72.5:0.0
80.411..
61.01:0.6
20.4
50.91:2.1
71.6:0.8
65.3=1.1
72.5:0.3
LAME+DART (ours)
78.710.7
70.7.0.9
818_0.
95.310.77
710111
78.511.11
59.810.4
11,1119
78.6..07
70.010.7
78.8.3.44
76.010.2
DELTA
78.62110
98 6:2.2
75.4-22
59.5-02
$0.9:122
77.051.4
720=1.1
02/2-5.0
78.220.6
DART (ours)
81.7+1.1
08-3+0.1
77.0-1.3
89.8-0.3
07.7-0.2
80.5+1.6
7..5+1.2
76.7+0.9
66.6+118
63.6+00.4
76.7-2.1
71.8+1.1
81.7+0.4
78.911.3
78.7.12
88.
98.110.3
78.410.8
82.810.7
76.910.8
50,9118
$0.212.1
73.410.8
65.011.0
17.410.5
ODS+DART (ows)
80.1213
38. 4:02
78 0
888202
97.8-03
77.8:0.8
81.42015
70.0:1122
713216
7492003
7.22219
79.720.4
SAR
78.6+0.8
97. 3-0
70.3-1.6
834-0.4
95.8+0.6
74.9+2.1
78-0+0.7
67.8+0.3
46.40088
62.6+0.5
72.5-0.5
30.7-0.6
74,2+0.4
SAR+DARTT (ours)
80.310.6
11.0
7099199
86.7.04
96.2.10.2
71.211.1
76.6.0..7
66.6.0..0
55.111.3
76.5.0.44
80.3.0.44
78.6.0.3
78.5.00.2
41

---

Published in Transactions on Machine Learning Research (02/2025)
Table 23: Average accuracy (%) on OfficeHome. X2Y refers to a scenario where a pre-trained model, trained in domain X, is tested in domain Y. For example, a2c refers to a scenario where a pre-trained model, trained in the art domain, is lested in the clipart domain.
Meshod
x2p
<lp
p2a
pdc
p2r
x3p
awg
NaAdapt
47.9
G5 0000
73:2:
524=0.0
63:2=000
64:200.0
40.8:0.0
46.3=100
7188+11.1
60.220.0
51.7=0.0
77.5=00.0
600.00000
RNAdspt
48.2+0.1
62. 0..2
71.4-02
51.2-0.3
02.6+0.2
62.9+0.1
51.3+0.1
48.3+0.4
72.3+0.4
64.5+0.44
53.6-0.5
76.3+0.2
60.4+0.1
BNAdupt-DATE -DAIT (oun)
47.810.3
5..011
024
52.7.044
63.210.2
63.710.2
51.9....5
38.010.2
72.8.0.4
64.5.0.4
53.8.0.4
76.8.0.2
61.010.1
TENT
48.120.4
02 020.0
03.5
54.6=044
63.720.7
60.320.8
52.2503
47.740.77
72.44115
65.5211.7
54.00066
70.6-0.4
00.920.22
TENTS DART (ours
48.8+0.4
06.3+0.5
7144-04
65.8-05
65.0+0.6
65.7+0.4
03.3+0.4
48.4+0.4
72.8+0.3
66.0+006
64.8-077
77.2-0.4
62.1+0.2
PL
66.310.
60.310.1
66.7:06
51.5:04
61.710.5
62.81002
51.110.1
16.11112
71,2104
64.710.5
51.7104
75.0.05
50.210..
PL+DART (ours]
40.8.0.4
02 520.77
71.0.033
519-111
62.12:033
63.7200.6
51.1105
46.811.7
64.71...5
52:121.5
75.120.4
00.110
NOTE
4111+0.9
41.4
G1.4 4-03
37.7=13
16.1=100
01.2=00.8
46.5+11.1
43.2+133
62.7+0.5
00.5+00.5
48.3-1.7
00.7=111
03.7+0.3
NOTE-DART (ours)
1..42:.5
50.0::10
0.7
48
4
42,31224
62:7:006
60:0:1.1
18.6:111
60:3=110
53.5+0.1
LAME
40.5.0.5
00/210044
68.3.004
51.7-001
62.210..5
62.21.011
51.910.3
48.0100.4
64.811022
50.5.0.3
73.9.0.11
59.810.1
LAME DART (ours)
40.7=00.2
G1.3=0022
0844004
03:2=06
62.8=022
62.9=:011
DA.T=1144
00.0=1044
647700.1
63.8=0.4
70.0=0.3
61) 00.2=002
DELTA
30.6+0.6
65.66-0.4
72.3-044
56.3-09
67.1-0.7
62.8+0-77
40.9+0.4
66-6+1.0
56.7-0.5
78.1-0.3
63.1+0.1
DELTA+DART' (ous))
51.510.4
67.610022
7..6.03
57.
67.
67.61...6
50,210.4
50,1100
75:1191
66.910.7
56.910.7
78.3.0.1
63.010.11
ODS
43.5:0.5
GL.G20.7
70:2=04
519=03
63.1=007
00.250.3
09.52106
06.551666
55.9005
78.120.3
62.1=002
ODS-DAAT (ours
48.9+0.4
64.8+0.9
71.0-04
554-00
658-006
65.7+0.2
03.4+0.2
40.8+0.6
73.7+02
06.7+0.5
56.0-0.5
78.3-0.3
62.5+0.2
SAR
40.8.0.3
62 810.
712-.03
52.5.06
64.1103
61.910.1
51.210.5
30.110.3
722104
65.010.3
54.1105
76.6.0.2
61.1.0..0
SAR+DART (ours)
49.120.4
613364
724
5333003
613203
05.820.3
52
00.320.
72.721155
649400
54.700.4
77.120.22
61.720022
Table 24: Average accuracy (%) on CTFAR-100C-imb (TRI)
Mateel
lepabo_
Com Mar
Mar
Drock
plrietr
Nuldke
13.32..1
174482
186522
155uns
903084
12.4277
255224
617A82
158250
6-1400
(45-01
07:341
TENT
55 4
59:40:
15
715-63
01341
15
155+65
960-09
060-03
TEST- RESTE-DAAEE-UR (seza)
15.1284
513115
G19111
133485
512438
563183
SAB-- -p (oadd)
115613
(2438T
012234
023485
56:1100
13.2103
0222015
643207
511104
6.1a8
Table 25: Average accuracy (%) on CIFAR-100C-imb (IR200)
No
17:340
712-115
43 0344315
304-11
18l-kke BAR (een)
011.34
-12414
30051-3
TENT
263017
257484
363033
SAB
28346.7
313.05
253113
253035
318485
GDS
11261.7
510003
42312.
05361
37.305
610100
162-11
3241133
543186
72SALE
006115
233484
Table 26: Average accuracy (%) on CIFAR-100C-imb (IR500)
Mitled
Ixpebr
ddona
Mer
3000L 140r
1ng
Tecatcoom
dark insdes
17.220.5
17.8534
28.1315
15.2511
193230
173223
353293
11221.3
15:1:0d
367311
DART [rae]
135413
472415
5166333
1154E6
361.61
TENT- DAAT qitk (wes))
653416
851435
583415
5:3423
235416
034155
81-11
36340
165313
185230
SAR- DART
473415
175428
075-43
OUS
253200
03:210
424228
02230
DUS--D0RU-pAA (sers)
433210
468111
84T:11
061-14
053124
135518
815223
0L143A
4131331
033-111
07.6111
Table 27: Average accuracy (%) on CIFAR-100C-imb (TR50000)
Manical
gremias 60w
tndor
lepalo rodo
ges Mar
Trok
Telgkrone
piristr
[pg orepenike
Noldoe
13.120
12:3582
142522
023388
215133
0850R2
514313
617A83
153255
13-611
31482
60-02
60-13
11495
44482
60661
TENT
30e07
T3-03
63-03
73455
T6-61
25485
31-03
TG-01
03485
65461
76460
1857- BESTE-UAAE--U (seza)
J0313
543185
4111.
184318
47.1244
583129
43217
01137
77404
17-05
74465
SAR--DAXT-ph frav
432200
114222
4631.3
032213
017220
83322
S11118
633M
3321109
01115
41300
10419
311237
37:-14
44111
Table 28: Average accuracy (%) on ImageNet-CCimm C-imb (IR1)
03-01
115661
12:091
13 50:40
355-31
51491
1Tp-41
300401
TESeAkee aAR (eee)
153461
105451
153401
56500
255-01
31 22.4
53 33540
803-3
653401
155eC1
411-0
(5940
25518.7
123131
353633
383335
18.113
026681
111137
SARL
23226.1
255131
247413
3721n1
0/4.2
Sclant
SABL -A (sau))
63181
333281
111633
11115.2
DATT-A (s420)
23770.5
325431
322/45
257
253111
414102
2401
853885
553181
173436
947.14
QUUAL
540452
42

---

Published in Transactions on Machine Learning Research (02/2025)
Table 29; Average accuracy (%) ImageNet- C-imb (IR1000)
Mitted
QUAMEAS BGMM
Gan Mar
aothe
Drot
54
traade
powate
1PAL saapepdeee
0340 D
3.000
5440
17548.1
105001
1F3401
525+01
36503.
53.40
1T2-00
30000.3
21.4405
1822811
23:2811
013332
2-1-141
ERART
0.3-0
013483
53
355-13
11546.1
36340
6.838 5
13006
103055
63234
137237
267-64
227118
123380
823211
383182
24381
2534314
15526.1
134233
22226
282.0
20381
305an1
163233
145638
SAR--DAREE -Ap (ome
1332202
151224
16R382
143283
222281
22232
200011
163233
443187
142233
333344
383436
142263
182212
342384
285225
40330
301112
303232
0L2182
43393
03644
31,3112
082226
Table 30: Average accuracy (%) on ImageNct- C-imb (IR5000)
dibe
65611
65481
1546.1
15461
07431
62420
12548.
12648.1
1246.1
11:14611
15.3411
33648.1
BIGA tittttiahh lean)
4218.1
433133
L2382
192011
15201
72512
123007
123003
142386
303333
3.3230
135511
353380
10044
251133
TENT- CAAE
33463
3.1635
11400
30433
1E5467
55a13
595452
115402
SAR
18181
03333
63381
102011
13301
A3312
133082
112000
19500
184333
12328.1
155555
125303
100002
3233322
DUS
04383
63214
72203
03811
155612
01333
4.5157
-22322
261384
311219
UES-TAART (war))
73465
63434
63403
10460
16463
63433
113003
13566
155457
-5460
564410
355-1
316411
Table 31: Average accuracy (%) on ImageNet- C-imb (IR500000)
Mitted
pusoine zobe
sket. E:0M
Ingrém 25.
(bber
Mar
de Me
mo9e Mer
soen Mer
Imxt
connet
<batz caeoee
19SL Insprruiee
2148.
21811
21.03
-6101
12481
24331
37431
156E1
44120
51.01
62481
11400
42311
414E.1
41011
ADAII--ppt (ren)
TEXT
48488
0.6631
18400
12400
02431
15430
18681
12420
32483
13458
LA42:
LS1E1
LA353
11151
22233
13882
13500
13500
23013
33237
13507
725-1
153533
193333
135500
57525
393531
SAR4 LAt (cem))
18582
33111
17261
48311
101100
13200
154510
183133
220213
33522
121109
112504
330811
00S
10350
01031
02903
13410
13300
37811
134200
13000
58521
125884
399167
3400
113300
16:2017
331810
43


Reviews:
[{"review_id": "kWwDZC15Sq", "reviewer": "Reviewer_dzaf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n+ The proposed label Distribution shift-Aware prediction Refinement for Test-time adaptation (DART) can be used to improve the TTA under the label distribution shift setting, broadening the application scenarios of the TTA strategies. It may interest some TMLR audiences. \n+ The refinement module and the prediction deviation have some merits that may motivate follow-up researchers to design some new TTA methods.\n+ The analysis using the toy example in Sec. 2 helps to illustrate the motivation of the proposed method, e.g., the prediction deviation metric, and how it works.\n+ The experiments are extensive, with the ablation study to verify the effectiveness of the key components in DART, and further experimental analysis to justify the design of DART.\n+ The overall performance is better than the other SOTA methods on various datasets.\n+ Many implementation details and in-depth analysis (e.g., theoretical analysis) are provided in the Suppl., which facilitates the re-implementation and further understanding of the proposed method.\n\nCons:\n\n1. In the Introduction, why can the model learn to adjust inaccurate predictions caused by label distribution shifts by experiencing several batches with diverse class distributions using the labeled training dataset before the start of test time? This statement is not easy to understand because the relationship between adjusting inaccurate predictions and the diverse class distributions is not presented. In addition, the necessity of doing it before the start of test time is not clarified.\n\n2. Can we simply use some methods specific for long-tailed distribution to address the class distribution shift stated in the paper, e.g., balanced batch-level sampling? To answer this question, please also consider comparing with some long-tailed methods.\n\n3. This paper adopts the balanced/diverse sampling strategy (IID or Dirichlet distribution) using training data. How much does this sampling strategy contribute to the performance (e.g., compared with removing IID or Dirichlet-based sampling)? \n\n4. Why intermediate time rather than directly train it during the training phase? In Eq. (1), why are the parameters of the trained classifier fixed? These two questions may share the same answer. \n\n5. What is the accuracy obtained by setting different $\\alpha$ values?\n\n6. Why not directly use DART-split as the standard pipeline for both the small-scale and large-scale datasets? What are the results of DART-split on small-scale datasets, like CIFAR-10-CLT, OfficeHome, and so on?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper analyzes Test-time adaptation (TTA) methods under label distribution shifts. Based on the analysis, this paper introduces label Distribution shift-Aware prediction Refinement for Test-time adaptation (DART), which refines the predictions by focusing on class-wise confusion patterns. DART trains a prediction refinement module that is intended to be used during test time to detect and correct class distribution shifts for better pseudo-label accuracy. Experimental results show that DART can correct inaccurate predictions caused by test-time distribution shifts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the questions in Cons and revise the manuscript accordingly.", "broader_impact_concerns": "There is no concerns on the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YdfRXuc3Y1", "reviewer": "Reviewer_oJqT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "#### Strengths\n\n1. The paper addresses an important problem of test-time adaptation (TTA) under arbitrary label distributions.\n2. The motivation behind the proposed method is clearly clarified both theoretically and experimentally, highlighting its ability to correct inherent classification confusion patterns caused by domain and label shifts.\n3. The experimental results are robust, showing consistent improvements across multiple benchmarks, with stable gains over existing TTA methods.\n\n#### Weaknesses\n\n1. Section 2 of the paper is somewhat difficult to follow. It would be helpful if the authors could include a symbol reference at the beginning of Section 2. Additionally, regarding the theoretical results in 'Impact of Label Distribution Shifts on TTA', where data distributions are assumed to be two-dimensional Gaussian, it would be valuable to visualize some sampled instances and the Bayesian-optimal decision boundary in the 2D plane to directly demonstrate the inequalities in the conclusions.\n2. There is a lack of some necessary analysis for the proposed method. \n   1. **Convergence speed**: The method requires multiple training iterations (at least 50 epochs), increasing the computational load. Would reducing the number of training epochs lead to a significant drop in performance? \n   2. **Generalization to unseen label distributions**: Can the method adapt to label distributions that were not seen during training, especially in highly imbalanced cases? For example, if the model is trained on distributions with an imbalance ratio below 50, can it generalize to a test set with a higher imbalance ratio, such as 100, and vice versa?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper attempts to address performance degradation in test-time adaptation (TTA) when the test label distribution becomes im-balanced. The authors analyze the confusion matrix of TTA under label distribution shifts and highlight that inherent misclassification patterns contribute to poorer performance. They propose a metric to estimate the imbalance ratio of the test label distribution and introduce a new TTA method, DART, which leverages this metric and the original prediction distribution to refine the predictions. Empirical results on multiple benchmarks (CIFAR-10C, PACS, OfficeHome, and ImageNet) demonstrate significant improvements in accuracy for imbalanced test batches while maintaining stable performance when no label distribution shift occurs. Additionally, DART can easily be integrated with existing TTA methods designed for covariate shift.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add a symbol reference table.\n2. It would be beneficial to include an illustration of the propositions in 'Impact of Label Distribution Shifts on TTA'\n3. Experiment on the relationship between performance and training epochs.\n4. Clarification on the imbalance ratio selection of label distribution during training and performance on unseen label distributions.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hyHvkDplQU", "reviewer": "Reviewer_tTqA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The paper is clearly written, well motivated and contains convincing experiments on multiple datasets and combinations with many TTA methods. Multiple ablation studies confirm the importance of the various elements of the proposed solution.\n\nWeaknesses: It is not a strong weakness, but since the observed degradation of performance when using BNAdapt is mostly due to severe shifts, the motivation can be seen as limited with limited expected gains in mild cases.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors tackle the problem of TTA (Test-Time Adaptation) when there are severe class distribution shifts. They introduce a distribution-aware shift module that is trained using artificially created unbalanced batches of training images.\nTheir module can be used in combination with most existing TTA approaches, leading to consistent gains in accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think the paper is ready to be published as is. There are already numerous experiments in the supplementary material that address most of the concerns that a reviewer could have.", "broader_impact_concerns": "There are no broader impact concerns for this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Based on the above comments and the consensus among the reviewers, I would recommend the paper to be accepted.

However, it is still necessary to include some discussions regarding the feasibility of the proposed method when it applies to large-scale models and datasets as well as when the labeled training dataset is not provided even in a form of condensed data. Especially, for the former case, the experimental results on ImageNet are provided in Table 2, however, the proposed TTA method seems to perform worse than no adaptation when the imbalance ratios are high.
Decision: Accept (poster)