Noisy Test-Time Adaptation in Vision-Language Models
Authors: Chentao Cao
Zhun Zhong
Zhanke Zhou
Tongliang Liu
Yang Liu
Kun Zhang
Bo Han
Venue: ICLR 2025 Poster (2025)
Keywords: OOD Detection
Test-Time Adaptation
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Test-time adaptation (TTA) aims to address distribution shifts between source and target data by relying solely on target data during testing. In open-world scenarios, models often encounter noisy samples, i.e., samples outside the in-distribution (ID) label space. Leveraging the zero-shot capability of pre-trained vision-language models (VLMs), this paper introduces Zero-Shot Noisy TTA (ZS-NTTA), focusing on adapting the model to target data with noisy samples during test-time in a zero-shot manner. In the preliminary study, we reveal that existing TTA methods suffer from a severe performance decline under ZS-NTTA, often lagging behind even the frozen model. We conduct comprehensive experiments to analyze this phenomenon, revealing that the negative impact of unfiltered noisy data outweighs the benefits of clean data during model updating. In addition, as these methods adopt the adapting classifier to implement ID classification and noise detection sub-tasks, the ability of the model in both sub-tasks is largely hampered. Based on this analysis, we propose a novel framework that decouples the classifier and detector, focusing on developing an individual detector while keeping the classifier (including the backbone) frozen. Technically, we introduce the Adaptive Noise Detector (AdaND), which utilizes the frozen model's outputs as pseudo-labels to train a noise detector for detecting noisy samples effectively. To address clean data streams, we further inject Gaussian noise during adaptation, preventing the detector from misclassifying clean samples as noisy. Beyond the ZS-NTTA, AdaND can also improve the zero-shot out-of-distribution (ZS-OOD) detection ability of VLMs. Extensive experiments show that our method outperforms in both ZS-NTTA and ZS-OOD detection. On ImageNet, AdaND achieves a notable improvement of $8.32\%$ in harmonic mean accuracy ($\text{Acc}_\text{H}$) for ZS-NTTA and $9.40\%$ in FPR95 for ZS-OOD detection, compared to state-of-the-art methods. Importantly, AdaND is computationally efficient and comparable to the model-frozen method. The code is publicly available at: https://github.com/tmlr-group/ZS-NTTA.

Published as a conference paper at ICLR 2025
NOISY TEST-TIME ADAPTATION IN VISION-LANGUAGE MODELS
Zhun Zhong Kun Zhang Bo Han
Chentao Cao-
Zhanke Zhou!
Tongliang Liu
Yang Liu"
'TMLR Group, Department of Computer Science, Hong Kong Baptist University
Sccooo of Computer Science and Information Engineering, Hefei University of Technology
'Sydney AI Centre, The University of Sydney
*Computer Science and Engineering, University of Californa. Santa Cruz
Mohamedd bin Zayed University of Artificial Intelligence
*Carnegie Mellon University
ABSTRACT
Test-time adaptation (TTA) aims to address distribution shifts between source and target data by relying solely target data during testing. In open-world scenarios, models often encounter noisy samples, i.e., samples outside the in-distribution (ID) label space. Leveraging the zero-shot capability of pre-trained lasgune models (VLMs), this paper introduces Zero-Shot Noisy TTA (ZS-NTTA), focusing on adapting the model to target data with noisy samples during test-time in a zero-shot manner. In the preliminary study, we reveal that existing TTA methods suffer from severe performance decline under ZS-NTTA, often lagging behind even the frozen model. We conduct comprehensive experiments to analyze this phenomenon, revealing that the negative impact of unfiltered noisy data outweighs the benefits of clean data during model updating. In addition, as these methods adopt the adapting classifier to implement ID classification and noise detection sub-tasks, the ability of the model in both sub-tasks is largely hampered. Based on this analysis, we propose novel framework that decouples the classifier and individual detector while keeping the classifier
detector, focusing on developing (including the backbone) frozen. Technically, we introduce the Adaptive Noise Detector (AdaND), which utilizes the frozen model's outputs as pseudo-labels train a noise detector for detecting noisy samples effectively. To address clean data streams, we further inject Gaussian noise during adaptation, preventing the detector from misclassifying clean samples as noisy. Beyond the ZS-NTTA, AdaND can also improve the zero-shot muttrf-distribution (ZS-OOD) detection ability VLMs. Extensive experiments show that our method outperforms in both ZS-NTTA and ZS-OOD detection. On ImageNet, AdaND achieves a notable improvement 8.32% in harmonic mean accuracy (AccH) for ZS-NTTA and 9.40% in FPR95 for ZS-OOD detection, compared to state-of of the-art methods. Importantly, AdaND is computationally efficient and comparable the model-frozer method. The code is publicly available at: https: / /ggttub........ com/
commtmllr tml aalr-ggrous grruprrop///ooooooro NS-NTTA.
INTRODUCTION
Machine learning models suffer performance degradation when the target distribution differs from the source distribution. To mitigate this issue, test-time adaptation (TTA) (Wang et al., 2021; Niu et al., 2023; Wang et al., 2022; Gao et al., 2023a; Liang et al., 2023) has been introduced, aiming to enhance models' generalization to the target distribution in test-time. However, TTA assumes the labels of testing samples are within the in-distribution (ID) label space, which is not practical in an open-world setting (Yang et al., 2022: 2021) where models often encounter noisy samples'.
This paper introduces the Zero-Shot Noisy TTA (ZS-NTTA) setting, which leverages off-the-shelf pre-trained vision-language models (VLMs) (Radford et al., 2021) to adapt target data containing
*Corresoondence to Bo Han (bhanml @ comp. hkbu.cdu.hk) and Zhun Zhong (zhunzhong007 @ gmail.com). Noisy samples refer data that lie outside the ID label space, whereas clean samples stay within it.

---

Published as a conference paper at ICLR 2025
Training stage: Teek-specific
Test stage:
Training stage: Tesk-specific
Test stage:
ID Label Space
specific Model
Clean Acd
specific Model
Clean/ Notey Acc
Targel Label *x Space.
TTA Setting
Noley TTA Setting
I-
dlean sample
Training slage: Tack-spec
Test slage:
Training slage: Tack
Test stage:
noiy sample
X Need
Task- agnostic Model
Task- agnostic
Clean/ Noisy Acc
AUROC FPR85
>>
Model
Training tess time
Frozen test
Zero-shot DOD Detection Setting
Zero-shot Notey TTA Setting
Comparison between TTA, noisy TTA, zero-shot OOD detection, and the proposed zero- shot noisy TTA. Only zero-shot noisy TTA focuses on both clean/noisy classification accuracy and performs in task-agnostic/ zero-shot manner. ZS-NTTA requires online detection of noisy samples.
Figure 1:
noisy samples during test-time in zero-shot way. Different from Zero-Shot Out-Of-Distribttion (ZS-OOD) Detection (Ming et al., 2022; Esmaeilpour et al., 2022; Wang et al., 2023), ZS-NTTA requires detecting noisy samples online and emphasizes classification accuracy more. Recently, several works (Li et al., 2023b; Gong et al., 2023) have explored the challenge of noisy samples in TTA, which require speciieci models that are pre-trained with specific source datasets. However, Li et al. (2023b) requires prototypes of the training data, which are unavailable in VLMs. On the other hand, Gong et al. (2023) focuses solely on the classification of clean data, neglecting the recognition of noisy samples. The comparison of different settings is illustrated in Figure 1.
In this paper, we first build the ZS-NTTA benchmarks by leveraging CLIP the VLM and evaluate the perfor- mance of existing TTA methods. We equip each method with the advanced OOD detection technique (Ming al., 2022) and an adaptive threshold to filter out noisy sam- ples. Figure 2 shows the performance rankings of existing methods in 7S-NTTA across 44 ID-OOD dataset pairs. We find the zero-shot CLIP (ZS-CLIP), which frozen during adaptation, shows promising performance, partic- ularly in distinguishing between clean and noisy samples, Despite filtering out noisy samples before updating the model, most TTA methods still underperform ZS- CLIP.
Acch Ranks
3.
ZS-CLIP
SOTTA
Figure 2: Performance ranking distri-
We design three model adaptation pipelines to understand the above phenomenon and analyze the impact of noisy and clean samples gradients during adaptation. Our findings reveal that noisy samples commonly lead to much larger gradients, often by an order of magnitude, compared to clean samples. Therefore, for methods (Wang et al., 2021) that continuously optimize the parameters during the adaptation, the model is prone to overfitting to noisy samples. Furthermore, that reset parameters at each step, their ability to distinguish between clean and noisy samples will be diminished after each update with noisy data. This underscores the detrimental effect of unfiltered noisy samples on model adaptation, outweighing the benefits of clean samples. Moreover, since these TTA methods implement ID classification and noise detection sub-tasks with the adapting classifier, the ability of models to handle both sub-task will be significantly reduced. Thus, we raise question:
bution of five TTA methods across 44 ID-OOD dataset pairs. The ranks of dif- ferent methods on one ID-OOD pair are ranked according to accuracy Accn. A rank closer to 1 denotes better perfor- larger bottom area reflects superior overall performance. We also evaluate these methods using absolute accuracy in Figure 7 in Appendix G.
mance,
for methods (Shu al., 2022)
How to effectively detect noisy samples to mitigate their negative impacts in test-time adaptation?
To this end, we propose a novel framework inspired by the above observation, which decouples the classifier and detector with focus developing individual detector while keeping the classifier (including the backbone) frozen. This framework offers two key benefits: 1) better distinguishing between noisy and clean samples, and 2) preventing detrimental effects caused by the classifier
2

---

Published as a conference paper at ICLR 2025
adapting to noisy samples. Technically, we propose Adaptive Noise Detector, termed AdaND. Given that can effectively distinguish the most clean and noisy samples, we utilize data filtered by ZS-CLII to train detector while keeping the rest of the model frozen during the testing phase. When encountering clean data streams, the detector tends to misclassify numerous clean samples as noisy ones. To handle such a situation, we propose intentionally introducing Gaussian noise during adaption, leading to an effective detector that is robust to both clean and noisy scenarios.
AdaND offers several advantages: 1) Zero-shot: By leveraging off-the-shelf VLMs, AdaND can accommodate various ID datasets and scale effectively to ImageNet and its variants; 2) Noise-agnostic: AdaND can handle a range of noise scenarios, including various types of noisy samples and different noise ratios (including scenario with exactly clean data); 3) High-performamce AdaND exhibits superior performance ZS-NTTA. In addition, AdaND can extend to ZS-OOD detection task and produce state-of- the-t-- performance; 4) Low computational overhead: The computational cost of AdaND) is comparable to that of frozen CLIP. Our contributions can be summarized as follows:
We propose more practical setting, i.e., Zero-Shot Noisy TTA (ZS-NTTA), and build benchmarks for evaluation. Based on the built benchmarks, we analyze why adapted methods suffer from performance decline and underperform the model-frozen method in ZS-NTTA (Sec. & Sec. 3).
We propose AdaND, simple and effective method that can cover both noisy and clean data streams in ZS-NTTA, offering computational efficiency comparable to model-frozen method (Sec. 4).
Our method demonstrates superior performance both ZS-NTTA and ZS-OOD detection tasks. Notably, in ImageNet. AdaND achieves a 8.32% improvement in Accı compared to existing TTA methods and a 9.40% improvement FPR95 over current OOD detection methods (Sec. 5).
ZERO-SHOT NOISY TTA
Definition of In-Distribution in VLMs. Following zero-shot OOD detection (Ming al., 2022; Esmaeilpour et al., 2022; Jiang et al., 2024), in our setting, the in-distribution (ID) classes are defined based on the classification task of interest rather than the classes used in pre-training. Accordingly. noisy samples are defined as data outside the ID label space.
Problem Formulation. We define the test set D = [X', U Vaesy where X' indicates the input space, Di represents the ID label space, and Monisy denotes the noisy label space. We are given input samples {a;} E X, the ID class names )aa {yı, 1/2, -.... yKK } with K classes, and pre-trained VL.ss. Owing to being trained on vast amounts of data, VLMs have learned robust feature representations. thereby enabling classification in zero-shot manner. Due noisy samples in the test data stream, first detect whether an input sample is noisy. If the sample is identified as clean, it is classified using the VLM. It directly categorized without further classification if recognized
noisy sample.
Why Investigating ZS-NTTA is Meaningful and Practical. One cannot ignore the noisy samples in real-world TTA deployment since the real world is open and full of unknown samples. We have demonstrated that the noisy sample is significant obstacle existing TTA methods in Sec. 3.2. While ZS-OOD detection (Ming et al., 2022; Jiang et al., 2024; Esmacilpour et al., 2022) considers noisy samples, it primarily focuses on the model's detection capability rather than improving the classification capability for ID data. More critically, the ID classification in existing ZS-OOD detection methods is typically evaluated in a closed-world setting, assuming a clean data stream. In contrast, ZS-NTTA requires detecting noisy samples online, placing greater emphasis on classification accuracy in open- world settings. We also discuss and compare existing test-time OOD detection work (Fan et al., 2024; Gao et al., 2023b) in Appendix 1. What's 's more, leveraging VI.Ms, ZS-NTTA can be performed in a zero-shot manner, making it more practical than noisy TTA.
We use three metrics to evaluate the performance in ZS-NTTA: Accs, Acey, and AсCи. Accş measures classification accuracy on clean samples², Acex measures detection accuracy on noisy samples, and Accи is the harmonic mean of Aces and Accx. providing balanced measure of both accuracies. The specific formulations of these metrics are as follows:
Evaluation Protocol.
ENiMED 1(y, 8.) 1(y. E Dia) E
Σ Enj ED 1(9, )nsey) I(y, E Vasey)
Aucs Acee
Acds
Acon
Accu
EmpEE (wv Via)
Exmee 1(1, )asis)
Accs Aoox
Note that, if clean sample is recognized a noisy sample, it is wrongly classified.
3

---

Published as a conference paper at ICLR 2025
Simple Baseline. We introduce simple yet effective baseline named ZS- ES-CLIP, employing the MCM (Ming et al., 2022) score as our score function to evaluate the confidence of the model's output for detecting noisy samples. Following seoo OOD detection (Ming et al., 2022), we construct the classifier using ID class names and perform classification based on the cosine similarity between the input image feature [(x;) and text features (T(+))) k=1- We define the cosine similarity between ((4))
the image and text features as follows: (xx4)) = TZ [F Here, I denotes the image encoder, and T signifies the text encoder. represents the input sample, and & is the text prompt "this a photo of (yx)" corresponding to the class name yk- We can detect the input sample through the noise detector G(-):
Clean S(x,) 2 X
where S(x,) = max esa(2i)/T
Gx(z,)
( Noise S(x:) X
Σ1=1 e*s(ai)/T"
where À is the threshold, S(-) denotes the MCM score, and is the temperature. If the sample is detected as clean, we then use the text-based classifier to classify it.
Adaptive Threshold. Various ID datasets can be encountered in ZS-NTTA, making fixed thresh- old À suboptimal. Therefore, an adaptive threshold is a better choice. According OWTTT (Li et al., 2023b), the distribution OOD scores follows a bimodal distribution. Based on this observation, Li et al. (2023b) proposes minimizing intra-class variance to determine the adaptive threshold:
min
Nia Σ (S(a))
Σ (S(rz)) A)5(2;)1 Nood Eis(a:)
Noed &
S
No
(3) where Nu = Σ (((z)) > Noxi = Σ (((())) S X) and N₂ is the length of queue at test-time update the score distribution. However, the score in OWTTT relies on source prototypes, which are unavailable in pre-trained VLMs. Here, we propose using the MCM score as an alternative. Furthermore, we conduct experiments with various fixed thresholds ranging from 0.1 to 0.9 to validate the reliability of our adaptive threshold, as detailed in Appendix C. The averaged results across different ID datasets indicate that the adaptive threshold outperforms fixed threshold.
A COMPREHENSIVE ANALYSIS OF ZERO-SHOT NOISY TTA
In this section, we introduce our 7S-NTTA benchmark and provide a comprehensive analysis of the performance of current TTA methods for this task.
3.1 ZERO-SHOT NOISY TTA BENCHMARK
Benchmark Datasets. To prevent overlap in label spaces of noisy and clean samples, we use established ID-OOD dataset' pairs from standard OOD detection benchmarks. The ID datasets include CIFAR-10/100 (Krizhevsky et al., 2009), CUB-200-2011 (Wah et al., 2011), STANFORD- CARS (Krause et al., 2013), Food-101 (Bossard et al., 2014), Oxford-IIIT Pet (Parkhi et al., 2012), ImageNet (Deng et al., 2009), ImageNet-V2 (Recht et al., 2019), ImageNet-A (Hendrycks et al., 2021b), ImageNet-R (Hendrycks et al., a021a), and ImageNet-Sketch (Wang et al., 2019). The OOD datasets encompass SVHN (Netzer et al., 1), LSUN (Yu et al., 2015). iNaturalist (Van Horn et al., 2018), SUN (Xiao al., 2010), Places (Zhou et al.. 2017), and Texture (Cimpoi et al., 2014). The specific ID-OOD pairs are detailed in Table 8 in Appendix D.I.
Evaluated Methods. We evaluate ZS-CLIP (Radford et al., 2021), Tent (Wang et al., 2021). SoTTA (Gong et al., 2023), and TPT (Shu et al., 2022) in our benchmarks. ZS-CLIP keeps all parameters frozen and utilizes Eq. (2) to determine whether an input sample belongs to the clean or noisy set. Samples identified as clean, denoted as n/, are then subjected to further classification. The other methods also utilize Eq. (2) to filter samples, subsequently using 2! update the model. Specifically, Tent updates the normalization layers within the image encoder by entropy minimization. SoTTA stores z to memory bank and selects the highest confidence samples to update the model every 64 steps using entropy-sharpme minimization. TPT applies data augmentation to x and updates the text prompt through entropy minimization.
'ID datasets and clean datasets interchangeable, as are OOD datasets and noisy datasets.
4

---

Published as a conference paper at ICLR 2025
Table 1:
Failure case study of existing TTA methods with CIFAR-I0 as the ID dataset. Green indicates an improvement over ZS-CLIP in average Acen, while red indicates the opposite.
Method
SVHN
LSUN
Testure
Placess
Avg
Acts
Acea
Acts
Acce
Acea
Aces
Accy
Acta
Aces
Accs
Accal
Aces
Aoce
ZS-CLIP
78.
Tent (GT))
90.77
96.99
93.78
90.40
93.55
91.95
90.07
90.22
90.14
89.87
74.50
81.47
90.28
88.81
89.34 (+2.878)
Teat (Nommal)
87.18
52.90
65.85
89.03
73.96
80.80
89.78
$8.48
89.13
88.78
65.44
75.34
88.69
70.19
77.78 (-8.69%)
Teat (All (All-update)
81.74
13.13
56.47
80.17
55.59
65.65
89.28
$1.64
86.90
87.86
56.27
68.60
81.76
59.91
69.41 (-17.068))
SoTTA (Normal)
90.21
81.71
85.75
90.13
91.06
90.39
89.56
90.96
90.25
89.04
74.17
80.93
89.73
84.47
86.88 (+0.42%)
So TTA (All-update)
89.69
73.13
80.57
89.88
90.76
90.32
89.17
90.54
90.00
89.05
74.50
13
89.52
82.23
85.50 (5.50(-0.9660 (-0.96%)
TPT (CT)
85.86
98.44
91.73
85.86
98.00
91.53
85.19
92.30
88
84.88
77.33
80.93
85.45
91.52
88.30 (+1.73%)
(Normal)
81.53
80.43
79.88
78.51
80.90
(-0.75%)
TPT (All-opuatte)
85.18
96.98
90.70
84.81
91.15
87.88
83.92
75.36
79.41
83.59
54.11
63.69
84.38
79.40
80.92 (-5.559)
OOD
Dansity
Drensii
Scure
score dfTerenco
000 Scare
Dute scream Jlpp
(a) 7S-CLIP
(b) Tent
(c) Score difference
Figure 3: Failure case analysis of Tent (Wang et al., 2021) in ZS-NTTA. (a) and (b) show the score distributions of ZS-CLIP and Tent, respectively, revealing that Tent makes it difficult to distinguish between clean and noisy samples. The horizontal axis is the value of OOD score. (c) illustrates the score difference between Tent and ZS- ES-CLIP, indicating that the confidence of noisy samples tends to increase in Tent. ID dataset: CIFAR-10, OOD dataset: SVHN.
3.2 FAILURE CASE STUDY
In this subsection, we analyze the failure case illustrated in Figure 2, i.e., 7S-CLIP outperforms most tuning-based methods on most ID datasets, highlighting three key observations. We begin by introducing three designed model adaptation pipelines to illustrate the impact of noisy samples on model adaptation (Observation 3.1). Subsequently, we visualize the score difference between ZS-CLIP and tuning-based methods to understand the failure case (Observation 3.2). Finally, we delve into the underlying reasons for the significant negative impact of noisy samples on model adaptation by conducting analyses of the model's gradients (Observation 3.3).
Observation 3.1. Noisy samples have significant negative impact model adaptation during TTA.
To investigate the impact of noisy samples in TTA, we construct three pipelines for each fine- tuning approach: Ground Truth (GT), Normal, and A1 All-update pipelines. The GT pipeline updates the model parameters using only the ground truth clean data, which is unavailable in practice. The No rmal pipeline updates the parameters using the data filtered by Eq. (2), which may include some noisy data, and this is the pipeline adopted in our main results (Sec. 5). The All -ulduddtte pipeline updates the model parameters using all the available data, i.e., it includes all the noisy data.
Table presents the performance of the three pipelines using CIFAR-10 as the ID dataset, The performance hierarchy observed for most methods is GT > ZS-CLIP > Normal 1 >> All- aall-uddate This indicates that for the Normal pipeline, the negative impact of the unfiltered noisy data on model adaptation outweighs the benefits of the clean data, resulting performance inferior to that of ZS- CLIP. SoTTA is on par with ZS-CLIP within the Normal 1 pipeline due to its refined sample selection for model adaptation. SoTTA employs memory bank to store high-confidence samples, utilizing only those with the highest confidence samples for updating the model. This strategy effectively filters out the majority of noisy samples, aligning with our assertion that noisy samples significantly and negatively impact model adaptation. Nonetheless, the improvement of SoTTA over ZS-CLIP remains marginal. For failure cases involving more ID datasets, please refer to Appendix G.I.
Observation 3.2. Throughout the model adaptation process in Tent. the scores of noisy samples gradually increase, ultimately rendering the MCM score incapable of distinguishing noisy samples.
5

---

Published as a conference paper at ICLR 2025
Toul samples
Clasiifel
Avg gradeut
Total comples
Clasified
Avg griden
step
us cleas
alcan
stgee 1
2991
2.1044
3141
7.80-3
stage
3181
2973
1.504
1.
200
stege 3-
2697
1.4-4
3235
3060
1.0c4
300
-0010
-1.01
(1:00)
0.055
0010
40100
4.905
03000
0.005
DOlD
gradient
gradient
(a) Clean sample
(b) Noisy sample
Figure 4: The impact of clean and noisy samples on the gradients. Note that the gradients of noisy samples are substantially larger in the first and second stages. The model effectively filters out noisy samples in the first stage but gradually struggles to distinguish between clean and noisy samples. ID dataset: CIFAR-10 10; OOD dataset: SVHN: Batch size: 64. Please see Figure 9 for an enlarged view.
We show the score distributions for 7S-CLIP and Tent under the No rmal pipeline in Figures 3a and 3b to better understand the impact of unfiltered noisy samples on model adaptation. Additionally, Figure 3c depicts the score differences for the same input sample between Tent and ZS-CLIP. ZS- CLIP effectively separates ID and OOD score distributions. In contrast, the increase in scores for most noisy samples in Tent makes the distinction between clean and noisy samples difficult. For the analysis of TPT, please refer to Appendix G.2.
Observation 3.3. MCM score with the adaptive threshold can detect most noisy samples during the early stages of TTA in Tent, though some inaccuracies may remain. However, these few inaccuracies during the early TTA stages can gradually lead the model to overfit to noisy samples.
We analyze the model's gradients in Tent under the Normal pipeline to understand why noisy samples negatively impact model adaptation. Figure shows how clean and noisy samples affect the gradients of the final layer normalization in the image encoder during TTA. As for clean samples, the model's ' gradients gradually decrease and remain relatively stable. The impact of noisy samples on the model's gradients can be roughly divided into three stages.
First Stage: The model effectively filters out noisy samples, with only a minimal number being erroneously classified as clean samples.
The model's performance progressively declines the impact of noisy samples becomes more apparent. The reliability of the MCM score weakens, and the model increasingly struggles to identify noisy samples. Moreover, the gradient magnitude of the noisy samples remains significant during this stage.
Second Stage:
Final Stage: The model overfits to the noisy samples, resulting in decrease in the model's gradient magnitude. At this stage, almost loses the ability to distinguish between clean and noisy samples.
Note that TPT resets the model at each step, meaning noisy samples' influence on the model's updates does not be accumulated. As a result, the impact of noisy samples on TPT is relatively smaller compared to Tent. Nonetheless, learning with noisy samples, with model reset at each step, still results in TPT performing worse than ZS-CLIP.
To this end, we naturally consider whether decoupling the classifier and detector might be superior strategy for the ZS-NTTA task. On one hand, focusing on developing a robust detector can more effectively distinguish noisy samples. On the other hand, keeping the classifier frozen can prevent from the adverse effects of adapting to noisy samples.
4 METHOD
This section demonstrates how to develop the framework that decouples the classifier and detector to better cope with the ZS-NTTA task based on the analysis presented in Sec. 3.2. The proposed framework focuses on training an adaptive noise detector to distinguish noisy samples while keeping the classifier frozen. Specifically, our method consists of two modules: (1) an Adaptive Noise Detector (AdaND), and (2) intentionally injecting Gaussian noise to cover the clean data stream case. The overall framework is illustrated in 5.

---

Published as a conference paper at ICLR 2025
plane
T.
adugtiies threshole
singe1:
dog
photo of a (class).
Encoder
Text
T3
4.
OOD Soore
step
ID classification
bird
stage::
Ty
Tx
pasudc-label
stop >m
adaptisn threshold
Image Encoder
Noise Detector
4
Inject Gaussian noise image
ID sample
OOD sample
Caussian noise samp.a
Frozen
Training
Figure 5: Overview of the proposed framework. We use the detection results from ZS-CLI as pseudo-labels to train the Adaptive Noise Detector (AdaND). In the early stage, we directly use the ZS-CLIP to distinguish clean-noise samples, while in the later stage, predicted clean samples are then classified based on the text- based classifier. To further handle the clean data stream case, we intentionally inject Gaussian noise additional noisy samples to avoid wrongly assigning too many clean samples as noisy ones.
the AdaND instead. The
4.1 ADAPTIVE NOISE DETECTOR
We use the image feature I(x) extracted from the frozen model the training data during TTA. Given that ZS-CLIP can effectively distinguish most ID and noisy samples, we use the detection results from ZS-CLIP as pseudo-labed in test-time throughout the process. In addition, we employ a single linear layer as the noise detector, leveraging the standard cross-entropy loss for training, ie., L Σ = V/ log( (g)()), J1 eat "E e5.. Here, v is the pseudo-labe generated by ZS-CLIP 2y denotes the logit the noise detector for class i, and C = 2. Our computational overhead is low since only the noise detector is updated during training.
After each training step, the test sample will be re-evaluated for clean-noise detection and classification using its image feature. Since the noise detector may not adapt sufficiently in the early steps of the data stream, we divide the clean-noise detection process into two stages. In the first stage, e,g. for the initial N optimization steps, we use the output from ZS-CLIP the detection result. In the second stage, we switch to using the output from the noise detector as the detection result. We also use the adaptive threshold in Eq. (3) as the detection threshold rather than directly set À 0.5.
To handle scenarios involving a single input sample, i.e., the batch size is 1, we introduce a queue with capacity of store the outputs from the noise detector. We update the noise detector with the queue's data every L. samples, and empty the queue after each update. Note that each sample yields an immediate test result upon input and does not require the accumulation of samples. What's more, our queue only stores the input features, outputs, and pseudo-labed ensuring privacy while maintaining minimal and negligible computational and storage overheads.
4.2 GAUSSIAN NOISE INJECTING
How to handle the clean data stream without data stream prior? Although the noise detector effectively differentiates between noisy and clean samples within a noisy data stream, it encounters challenges when the test data lacks noisy samples. In such cases, the detector tends to misclassify many clean samples as noisy, leading to a significant drop in performance. To address this, we intentionally inject noise additional noisy samples to cover the clean data stream case. In this way, manually injected noise will be included in the adaptive threshold calculation, preventing misclassification of clean samples noisy. During testing, we exclusively consider samples from the original data stream, excluding manually injected noise samples.
How to choose the appropriate noise before inference? The injected noise must 1) lie outside the ID label space and 2) be easily accessible without incurring extra costs for auxiliary data collection. The choice of injected noise is flexible; for simplicity and effectiveness, we choose Gaussian noise.
7

---

Published as a conference paper at ICLR 2025
Table 2: Zero-shot noisy TTA results for CUB-200-2011, STANFORD-CARS, Food-101, and Oxford-IIIT Pet as the ID datasets. The bold indicates the best performance on each dataset.
ID
Method
INaturalis:
SUN
Texture
Places
Avg
Asis
Accy
Acdy
Aces
Accy
Addiy
Acdy
Addi
AcEn
Asts
Acdi
Actiy
Adiis
ZS-CLIP
88.06
37.56
50.94
53.04
52.59
Tent
37.02
46.95
41.40
38.61
55.55
45.56
34,98
41.77
38.07
40.41
74.83
37.75
54.78
44.38
CUB-200-2011
SoTTA
41.67
84.37
55.79
42.08
86.83
56.69
41.
77.38
54,02
42.04
86.52
56 56.59
41.81
83.82
55.77
IPT
37.41
89.57
52.78
37.49
89.67
52.87
36.88
51.67
50.81
37.44
89.45
52.79
37.30
$7.59
52.31
(Dui)
52.34
96.40
67.84
$2.41
93.91
67.27
51.82
81.34
63.28
51.82
91.51
66.17
52,10
91.77
66.14
7S-CLIP
96.62
65.05
53.48
98.81
53.59
69.55
53.36
98.05
69.1 11
98.13
68.53
STANFORD-CARS
SoTTA
$1.51
92.84
66.26
54.81
97.57
70.19
55, 15.06
70.64
54.75
96.96
60
$4.03
96.47
69:27
TPT
49.24
96.97
65.31
52. 40
98.83
49
52.75
09:22
68. 89
$2.42
98.39 39
68. 4n
$1.70
9W. 36
67:77
AGUND
62.90
99.79
77.09
62.73
99.82
77.04
62,91
99.75
77.16
62.76
99.29
76.91
62.80
99,66
77.05
ZS-CLIP
$0.60
94.76
87.11
$0.75
96.05
57.75
60.51
93.12
86.36
90.62
94.62
87.06
80.62
94.65
87.07
Tent
25.09
57.70
82.86
82.54
84.75
82
80.13
71.90
Food 101
SOTTA
87.58
62.05
91.62
86.79
85.80
94.93
85.65
79.92
96.19
30
79.70
93.86
86.20
79.76
95.14
86
79.77
95.03
86.73
AGAND
86.50
99.87
92.71
86,40
99.64
92.55
86.44
96.51
91.20
86.42
99.40
92.46
86.44
98.85
92.23
ZS-CLIP
78.58
88.30
63.16
79:75
87.30
$3.35
80.20
91.16
85.33
79.59
84.17
81.82
79.53
67.73
83.41
Teal
80.07
78.09
79.07
81.19
68.30
74, 19
81,48
74.72
77.95
90.64
62.51
70.43
80.84
70.91
75.41
Oxferd-IIT Pet
SOTTA
84.72
82.66
77.56
89.71
83.19
78.87
89.82
87.32
82.
78.56
AGaND (Ouns
85.81
98.78
91.84
82
98.1
91.39
85
98.68
91.82
83.88
96.58
90.92
85.84
98.06
91.54
During testing, we insert Gaussian noise sample for every M input sample in the data stream, regardless of whether the stream is clean or noisy. Note that we don't have prior knowledge about whether the data stream is clean or noisy. The detailed algorithm for AdaND is provided in Algorithm in Appendix D.2.
5
EXPERIMENTS
5.1 SETUPS
Compared Methods and Evaluation Metrics. We compare our method with existing TTA methods mentioned in Sec. 3.1 on the ZS-NTTA task using 11 ID datasets from Sec. 3.1, evaluating with Accs. Acen, and Acen. Additionally, we compare with leading OOD detection methods on the 7S-OOD task, including Energy(Liu et al., 2020), MaxLogit (Hendrycks et al., 2019a), MCM (Ming et al., 2022), CLIPN (Wang et al., 2023), and NegLabel (Jiang et al., 2024), using AUROC and FPR95 as metrics. Please see Appendix D.2 for the implementation details of compared methods.
AdaND Setups. In our main results, we maintain consistent hyper-parameters across all datasets. Specifically, we use CLIP (Radford et al., 2021) as our VLM, with ViT-B/16 (Dosovitskiy et al., 2020) the image encoder and masked self-attention Transformer (Vaswani et al., 2017) as the text encoder, both keeping frozen. We employ a single linear layer as our noise detector, which remains learnable throughout the TTA process. We optimize with Adam (Kingma & Ba, 2014), using a learning rate of 0.0005 and no weight decay. Gaussian noise is injected every 8 samples (M = 8). The noise detector's 's queue length (L) is set to 128, and the adaptive threshold's 's queue length (N₂) follows OWTTT (Li et al., 2023b) with a value of 512. We use N = 10 for the first stage. As for the ZS-OOD detection task, we use MCM (Ming et al., 2022) score from the output logit of the noise detector as our score function. Unless otherwise specified, we set the batch size (bs) to 1 for AdaND.
5.2 MAIN RESULTS
Zero-Shot Noisy TTA Task. Table 2 and Table 3 present a detailed comparison of ZS-NTTA task results across various ID datasets. On ImageNet, AdaND enhances the average performance by 8.32% in terms of AcCH- Although we filter the data using the MCM score and adaptive threshold, a considerable portion of noisy data remains unfiltered. Consequently, when Tent leverages the filtered data to update the model's normalization layers, it inadvertently causes a substantial performance decline. SoTTA improves data selection by focusing on the highest confidence samples, slightly outperforming ZS-CLIP on some datasets, but the gains are limited. Despite TPT resetting the model before each sample input, the unfiltered noisy data causes TPT to perform worse than ZS-CLIP on most ID datasets. Since our method decouples the classifier and detector, which focuses on developing the noise detector and keeping the classifier frozen, our AdaND can better identify noisy samples and prevent unfiltered ones from affecting the classifier. Due space constraints, the results for CIFAR- 10/100 are provided in Table 10 in Appendix E. In summary, our AdaND demonstrates superior performance over the compared methods, achieving the best results across all datasets.
8

---

Published as a conference paper at ICLR 2025
Table 3: Zero-shot noisy TTA results for ImageNet and its variants as the ID datasets. The bold indicates the best performance on each dataset.
ID
Method
INaturalist
SUN
Texture
Placess
Avg
Accy
Acda
Accs
Acty
Acce
Aucy
Acti
Acce
Avey
Asey
Acch
Aces
AsDo
Acce
ZS-CLIP
51,01
66.51
65.30
63.08
53.35
61.17
61.77
Tenl
48.56
35.74
41.18
55.44
75.54
63.95
54,94
70.93
61.92
55.76
73.98
63.59
53.67
64.05
$7.66
ImageNet
ScTTA
53.15
62.68
57.52
53.16
68.76
59.96
53.64
68.05
59.99
53.60
69.16
60.39
53.39
67.16
59.47
TPT
52.58
88.93
66.09
51.91
$6.09
61.77
51.11
80.01
62.38
51.80
82.89
63.76
51.85
B1,18
61.25
AdaND (Ours)
63.26
61.34
72.77
71.47
61.92
71.58
62.24
73.09
2S-CLIP
34.17
83,46
48.49
33.46
81.20
47.39
32.61
75.57
45.56
33.40
77.10
46.61
33.41
79.33
47.01
Tent
30.46
26.86
28.55
36.57
71.82
48.46
36.37
66.63
47.06
36.87
70.32
48.38
35.07
58.91
43.11
ImageNes-K
SoTTA
36.18
61.70
45.61
36.28
67.19
47.12
35.91
65.31
46.
36.57
67.09
47.54
36.23
65.32
46.60
TPT
32.16
46.89
31.55
45.85
30.74
77.39
44.00
45.27
45.50
AdIND (Ours)
40.97
93.54
56.98
40.25
85.06
54.64
38.31
74.43
50.58
39.66)
79.57
52.88
39.78
K3.15
53.77
ZS-CLIP
34.73
80.69
48.56
34, 20
78.83
47.70
33.97
76.60
47.07
33.96
75.11
46.77
34. 22
77.81
47.53
Tent
34,99
77.19
48.15
34.89
77.05
47.97
34. 36
75.19
47.17
34.60
73.83
47.12
34.70
75.81
47.60
ImageNet-A
SOTTA
36,85
76.83
49.81
36.47
77.08
49.51
601
75.37
48. 36
36.07
73,87
48.47
36.25
75.79
49.04
TPT
34.12
81.17
48.14
33.20
80.23
97
33.12
79.92
46.81
33.015
77.00
46.25
31.37
79.58
47.02
AdaND (Ours)
43,59
91.19
58.96
41.96
80.93
55:27
45,04
79.97
57.62
42.85
72.13
53.76
43.36
81,066
56.41
ZS-CLIP
60.38
59.92
Tent
47.94
76.98
59.08
48.28
80.50
60.36
47.56
74.47
58.05
48.34
77.37
59.50
48.03
77.33
59:25
ImageNet-V2 V2
ScTTA
48.24
78.59
59.78
47.80
78.67
59.47
47.27
74.82
$7.94
48.26
76.05
59.05
47.59
77.03
59.06
TPT
46.63
88.37
61.05
46.12
85.58
59.99 59.91
45.21
79.14
57.55
46.02
81,95
58.94
46.00
83.76
$9.37
AdaND (Ours)
56.32
97.06
71,28
54.78
67.12
57.28
66.97
55.81
65.49
56.05
67.72
ZS-CLIP
61.99
94.39
74.83
61,82
88.95
72.94
60.91
77.05
68.04
61.68
84.86
71.44
61.60
86.31
71.81
Tent
65.22
91.45
76.14
65.06
85.61
73.93
63.33
69.99
66.49
64.93
82.18
72.62
64.64
82.36
72.30
ImageNet-R
SoTTA
66.78
86.98
75.55
66.71
83.99
74.36
65.92
72.69
69.14
66.60
72:91
66.50
72.99
TPT
60.95
74.20
60.85
72.60
59.98
77.79
67.73
60.67
85.79
71.08
87.09
71.40
AdaND (Ours)
72.21
99.59
83.72
71.02
95.94
81.62
70.44
81.43
75.54
70.85
92.14
80.10
71.13
92.28
80.25
Table 4: Runtime and GPU memory with varying batch sizes (bs) on ImageNet for single sample.
Resource
ZS- CLIP (be = 1)
ScTTA (be
1)
TPT (be
Ours (be
ZS- ZS-CLIP (be 128)
Tont (ōs 128)
Ours (bs 128)
Time [S)]
0.1193
0.3219
0.1272
0.9015
0.00337
0.0017
Menory (GiB))
3,80
9.13
21.25
3.83
4,54
14.99
4,57
Table 4 shows the time and computational resources required to test single sample on the ImageNet. All comparisons were conducted on the same 80G A800 GPU. We tested 6, 400 samples and then averaged the results to ensure result stability. Since our method freezes the VLM and uses only single linear layer for the noise detector, our time consumption is nearly equivalent to ZS- ZS-CZIICC CLIP. Tent's 's result is reported with bs 128, performance drops significantly at bs = (See Table 1 1). TPT consumes the most time and memory due to its 64-fold data augmentation and gradient backpropagation through the entire text encoder. Our method proves to be more resource-efficient than Tent, SoTTA, and TPT.
Zero-Shot OOD Detection Task. The results for ZS-OOD detection are presented in Table 5, using ImageNet as the ID dataset. Our approach demonstrates competitive performance compared to tfate-of-the-an OOD detection methods, with significant improvements of 1.37% in AUROC and 9.40% in FPR95. Notably, CLIPN requires an additional dataset to train text encoder, and NegLabel needs to mine extra semantic information from a large-scale corpus database. In contrast, our method requires no additional external data, making simpler and more efficient. The results indicate that learning an adaptive noise detector is simple yet effective strategy for 7S-OOD detection task.
5.3 ABLATION STUDIES
The Effectiveness of Noise Detector and Injected Noise. We evaluate the effectiveness of the noise detector and Gaussian noise modules under both clean and noisy data streams using ImageNet as the ID dataset, as shown in Table 6. Without Gaussian noise, the noise detector alone is ineffective for clean data streams. When the noise detector is not present, the performance in noisy data streams decreases significantly. Our full method both effective under clean and noisy data streams, demonstrating the soundness of our design. Results for other ID) datasets are in Table 16.
Intentionally Injected Noise in AdaND. We conduct ablation studies on intentionally injected noise from two aspects: noise types (Gaussian, Uniform, Salt-and-pepper, Poisson) and injection frequency (every 2, 4, or 8 test samples). As shown in Table 17, all noise types effectively manage both clean and noisy data streams, demonstrating that our method is robust to the choice of injected noise. Table 18 shows the results for noise injection frequency using Gaussian noise. Our experiments show that injecting Gaussian noise every 2, 4, or 8 samples yields excellent performance. Considering efficiency and performance, we choose to inject Gaussian noise every 8 samples.
9

---

Published as a conference paper at ICLR 2025
Table 5: Zero-shot OOD detection results for ImageNet as the ID dataset. The bold indicates the best.
iNaturalist
SUN
Texture
Places
Avg
Method
AUROCT
FPR9S
AUROC
FPR95L
AUROC1
FPR9S
AUROCI
FPR951
AUROC1
FPR95]
Max-Logii
89.31
61.66
87.43
64.39
71.68
86.61
85.95
63.67
83.59
69.08
Energy
85.09
81.08
84.24
79.02
65.56
93.65
83.38
75.08
79.57
82.21
MCM
94.61
30.91
92.57
37.59
86.11
57.77
89.77
44.69
90.77
42.74
CLIPN
95.27
23.94
93.93
26.17
90.93
40.83
92.28
33.45
93.10
31.10
NegLabel
99,49
1.91
95.49
20.53
90.22
43.56
91.64
35.59
94.21
25.40
AdaND> (Ours)
98.91
4.19
95.86
17,08
93.01
21.76
94.55
20,95
95,58
16.00
Table Ablation studies for each module in the method using ImageNet as the ID dataset. Results in noisy data stream are averaged over four OOD datasets: indicates the exclusion of a module and '' indicates inclusion of a module.
iNaturalist, SUN, Texture, and Places. 'x'
Noise Detector
Gaussian Noise
Clean Data Stream
Noisy Data Stream
Accş
Acey
AccH
Accş
Acox
Acoæt
47.68
53.38
82.38
64.77
50.07
53.95
81.65
64.95
37.54
60.64
91.73
73.00
63.96
62.24
88.67
73.09
Simulating Real-world Adaptation. We simulate real-world adaptation by mixing ID and OOD datasets from two perspectives. The first involves varying noise ratios (0%, 25%, 50%, 75%) to mimic real-world conditions. The second considers the order of ID and OOD samples, which simulate using different random seeds. Table 19 presents the results for data streams with varying noise ratios. Since cannot assume prior knowledge of whether data stream is clean contains noisy samples, all methods retain an adaptive OOD detection threshold module. As a result, comparative methods exhibit significant performance degradation clean data streams. In contrast, our method, which deliberately injects Gaussian noise, effectively addresses clean data streams. Moreover, as the proportion of noise in the data stream increases, most comparative methods show a marked decline in performance, whereas our method continues to deliver strong results across different noise ratios. The experimental results for different random seeds are provided in Table 20 and Table 21. Due to computational constraints, we only conduct experiments using CIFAR- 10 and CIFAR- 100 as II) datasets, with random seeds ranging from 0 to 4. The results demonstrate that our method consistently achieves superior performance, regardless of the input order of the data streams.
Selection in AdaND. We conducted ablation experiments in AdaND with varying queue capacities L (32, 64, 128, 256, 512). As shown in Table 22, our method demonstrates insensitivity to the choice of L,, and we selected I, 128 for the main experiments Table 23 presents the ablation studies on the queue length Ng. which used update the score distribution for determining the adaptive threshold. Similar to L, AdaND is also robust to changes in N₂-- and following ОWTTТ, we set N, = 512. The results for different values Table 24. We found that N 10 optimization steps are sufficient to initialize the noise detector. In summary, AdaND exhibits low sensitivity to hyper-parameter settings across all datasets, which yields the best results compared to other methods.
parame-arrmeeers
are shown in
selection, allowing us to use consistent
We explore the performance of using different backbones in Table 25, Our AdaND is significantly better than the other methods when using different backbones. We also discuss using pseudo-labels labels generated by the noise detector as pseudo-labels in Appendix F.4. Using noise detector outputs as pseudo-labels can improve performance on some datasets but cause intolerable drops in others.
CONCLUSION
In this paper, we introduce the Zero-Shot Noisy TTA (ZS-NTTA) setting and construct benchmarks for evaluating this task. We investigate why existing TTA methods fail this setting by designing three model adaptation pipelines, visualizing the score difference, and analyzing the gradients to understand the impact of noisy samples on model adaptation. Based on these analyses, we propose AdaND, which decouples the classifier and detector, focusing on developing while keeping the classifier frozen. Our AdaND can handle both noisy and clean data streams by intentionally injecting Gaussian noise, preventing the noise detector from misclassifying excessive ID samples as noise during adaptation. Empirically, our method achieves state-of- ofte--of-the-rt results in both ZS-NTTA and ZS-OOD detection tasks. Moreover, our approach is computationally efficient.
adaptive detector
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
CTC, ZKZ, and BH were supported by RGC Young Collaborative Research Grant No. C2005-24Y, NSFC General Program No. 62376235, Guangdong Basic and Applied Basic Research Foundation Nos. 2022A1515011652 and 2024A1515012399 HKBU Faculty Niche Research Areas No. RC- TNRA-IGG22-22SSCI004 and HKBU CSD Departmental Incentive Scheme. TLL was partially supported by the following Australian Research Council projects: F1220100318, DP220102121, L.P20100527,, L.P22220999,, and IC190100031.
ETHICS STATEMENT
This work does not involve potential malicious or unintended uses, faimess considerations, privacy considerations, security considerations, crowdsourcing, or research with human subjects.
REPRODUCIBILITY STATEMENT
We provide details to reproduce our results in Sec. 2, Sec. 5.1, and Sec. D. We also provide pseudo-code in Algorithm 1, and the code is publicly available at: https:/
imir-group/2S- 2STNTTTA.
REFERENCES
Julian Bitterwolf, Maximilian Mueller, and Matthias Hein. In out? fixing imagenet out-of- distribution detection evaluation. arXiv preprint arXiv:2306.00826, 2023.
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101-mining discriminative compo- nents with random forests. In ECCV, 2014.
2014.
Malik Boudiaf, Romain Mueller, Ismail Ben Ayed, and Luca Bertinetto. Parameter-free online test-time adaptation. CVPR, 2022.
Chentao Cao, Zhun Zhong, Zhanke Zhou, Yang Liu, Tongliang Liu, and Bo Han. Envisioning outlier exposure by large language models for dust---distiniution detection. In ICMI,, 2024.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. ICML, 2020.
Mircea Cimpoi. Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In CVPR, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't ' know by virtual outlier synthesis. In ICLR, 2022.
Xuefeng Du, Zhen Fang, Ilias Diakonikolas, and Yixuan Li. How does unlabeled data provably help out-of-distribution detection? arXiv preprint arXiv:2402.03502. 2024.
Sepideh Esmaeilpour, Bing Liu, Eric Robertson, and Lei Shu. Zero-shot out-of-distributiin detection based the pre-trained model clip. In AAAI, 2022.
Ke Fan, Tong Liu, Xingyu Qiu, Yikai Wang, Lian Huai, Zeyu Shangguan, Shuang Gou, Fengjian Liu, Yugian Fu, Yanwei Fu, et al. Test-time linear out-of-distribution detection. In CVPR, 2024.
11

---

Published as a conference paper at ICLR 2025
Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? In NeurIPS, 2022.
Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning. In ICCV, 2023,
François Fleuret al. Test time adaptation through perturbation robustness. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
Jin Gao, Jialing Zhang, Xihui Liu, Trevor Darrell, Evan Shelhamer, and Dequan Wang, Back to the source: Diffusion-driven adaptation to test-time corruption. CVPR, 2023a.
Zhitong Gao, Shipeng Yan, and Xuming He. Atta: anomaly-aware test-time adaptation for out-of- distribution detection in segmentation. In NeurIPS, 2023b.
Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. SoTTA: Robust test-time adaptation on noisy data streams. In NeurlPS, 2023.
Shurui Gui, Xiner Li, and Shuiwang Ji. Active test-time adaptation: Theoretical analyses and an algorithm. In ICLR, 2024.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. In ICLR, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and examples in neural networks. In ICLR, 2017.
Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. arXiv preprint 1wXiv11911.11.. 2019a.
Dan Hendrycks, Mantas Mazeika, and Thomas Dictterich. Deep anomaly detection with outlier exposure. In ICLR, 2019b.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribttion generalization. In ICCV, 2021a.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, 2021b.
Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. In ICML, 2022.
Rui Huang and Yixuan Li. Mos: Towards scaling out-of-distribution detection for large semantic space. In CVPR, 2021.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021.
Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han. Negative label guided ood detection with pretrained lision-language models. In ICLR, 2024.
Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing. Efficient test-time adaptation of vision-language models. In CVPR, 2024.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV Workshops, 2013.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
12

---

Published as a conference paper at ICLR 2025
Jonghyun Lee, Dahuin Jung. Sachyung Lee, Junsung Park, Juhyeon Shin, Uiwon Hwang, and Sungroh Yoon. Entropy is not enough for test-time adaptation: From the perspective of disentangled factors. In ICLR, 2024.
Jungsoo Lee, Debasmit Das, Jaegul Choo, and Sungha Choi. Towards open-set test-time adaptation utilizing the wisdom of crowds in entropy minimization. In ICCV, 2023.
Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint St1i:22311.031999 2023a.
Yushu Li, Xun Xu, Yongyi Su, and Kui Jia. On the robustness of open-world test-time training:
Self-training with dynamic prototype expansion. In ICCV, 2023b.
Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey test-time adaptation under distribu- tion shifts. arXiv preprint arXiv:2303. 15361, 2023.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In NeurIPS, 2020.
Xiaosong Ma, Jie Zhang, Song Guo, and Wenchao Xu. Swapprompt: Test-time prompt adaptation for vision-language models. In NeurIPS, 2023.
Yifci Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of- distribution detection with vision-language representations. NeurIPS, 2022.
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D' Ammour Balaji Lakshminarayanan and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006. 10963, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. NeurIPS Workshop, 2011.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In ICML,, 2022.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation dynamic wild world. In ICLR, 2023.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, 2012.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-perfore peshorerformncc deep learning library. NeurIPS, 2019.
Mihir Prabhudesai, Tsung- Wei Ke, Alexander Cong Li, Deepak Pathak, and Katerina Fragkiadaki. Diffusion-tta: Test-time adaptation of discriminative models via generative feedback. In NeurlPS, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML,, 2021.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In ICML, 2019.
Jameel Hassan Abdul Samadh, Hanan Gani, Noor Hazim Hussein, Muhammad Uzair Khattak, Muzammal Naseer, Fahad Khan, and Salman Khan. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization. In NeurIPS, 2023.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann. Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In NeurlPS, 2020.
13

---

Published as a conference paper at ICLR 2025
Vikash Sehwag, Mung Chiang, and Prateek Mittal. Ssd: A unified framework for self-supervised outlier detection. In ICLR, 2021.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In ICML, 2023.
Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization vision-language models. In NeurIPS, 2022.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In CVPR, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd 200s-2200-0011 dataset. 2011.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021.
Haohan Wang. Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. NeurlPS, 2019.
Hualiang Wang, Yi Li, Huifeng Yao, and Xiaomeng Li. Clipn for zero-shot ood detection: Teaching clip to say no. In ICCV, 2023.
Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In CVPR, 2022.
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does Ilm safety training fail? In NeurlPS, 2023.
Jianxiong Xiao, James Hays, Krista Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey zoo. In CVPR, 2010.
Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, 2022.
Jingkang Yang. Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of dittrf-distribution detection: A survey, arXiv preprint arXiv: 10. 11334, 2021.
Jingkang Yang, Pengyun Wang. Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of- distribution detection. In NeurIPS, 2022.
Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark A. Hasegawa-Johnson. Yingzhen Li, and Chang D. Yoo. C-TPT: Calibrated test-time prompt tuning for vision-language models via text feature dispersion. In ICLR, 2024.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: construction of a large-scall scale image dataset using deep learning with humans in the loop. arXiv 2015.
preprint arxiv: 1.X.vv1506.033.
Shuai Zhao, Xiaohan Wang, Linchao Zhu, and Yi Yang. Test-time adaptation with CLIP reward for zero-shot generalization in vision-language models. In ICLR, 2024.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo, Zengmao Wang, and Bo Han. Can language models perform robust reasoning in chain-of ofothhught prompting with noisy rationales? In NeurIPS, 2024.
14

---

Published as a conference paper at ICLR 2025
APPENDIX
Discussion
16
A.1 A Further Discussion ZS-NTTA Setting.
16
A.2 Limitation
16
Related Work
17
C Adaptive Threshold vs. Fixed Threshold
18
D Experimental Details
18
D.I Dataset Details
18
D.2 Implementation Details
19
D.3 Environment
19
Additional Results
19
Full Results of Ablation Studies
22
F.1
Ablation Studies the Modules of AdaND
22
F2 Ablation Studies on the Intentionally Injected Noise
22
F3 Ablation Studies on Simulating Real-world Adaptation
22
F4 Ablation Studies on Hyper-parameters Selection
22
G Full Results of Failure Case
24
G.1 Three Model Adaptation Pipelines
24
G.2 Score Difference
25
G.3 Gradient Analysis
26
15

---

Published as a conference paper at ICLR 2025
A DISCUSSION
A.1 1 A FURTIER DISCUSSION ON ZS-NTTA SETTING
We have elaborated on the distinctions between ZS-NTTA and ZS-OOD detection in Sec. 2, which primarily lie in task objectives and evaluation settings. These differences also apply to the comparison between ZS-NTTA and test-time OOD detection, as the latter essentially shares the same objective as classical OOD detection. We further summarize the task definition differences between ZS-NTTA and Fan et al. (2024); Gao et al. (2023b) in Table 7. In this section, we also discuss and compare existing test-time OOD detection works (Fan et al., 2024; Gao et al., 2023b) regarding methodology. RTL (Fan et al., 2024) used linear regression to make a more precise OOD prediction. In other words, RTL leverages the TTA method to enhance OOD detection while fundamentally remaining an OOD detection task. Different from RTL, we focus on the TTA setting itself, where test samples may contain noise, resulting in severe performance degradation of existing TTA methods. ATTA (Gao al., 2023b) primarily addresses dense OOD detection in semantic segmentation; however, ATTA cannot be extended to the ZS-NTTA setting since it relies on measuring the distributional distance between test and training features in the normalization layers of the segmentation network. In the context of pretrained VLMs like CLIP, we don't 't have access to the training data, making ATTA's approach inapplicable to our setting.
In the era of Foundation Models (FMs), we believe noisy TTA can be further explored. The input to FMs may encompass diverse types of noise, including irrelevant or erroneous information (Zhou et al., 2024; Shi et al., 2023), as well as malicious prompts (Wei et al., 2023; Li et al., 2023a), which can significantly undermine the reasoning capabílities of FMs. How to address these noise inputs during testing while enhancing the reasoning capabilities of FMs is an important research direction.
Table 7: Comparison between ZS-NTTA and test-time OOD detection setting (Fan et al., 2024; Gao et al., 2023b).
Fan et al. (2024)
Gao et al. (2023b)
ZS-NTTA
Focus ID classification Focus on OOD detection
Evaluate ID classification
Clean data stream AUROC, FPR95
Clean data stream AUROC, FPR95
Noisy data stream
Metrics
Harmonic mean accuracy (AccH)
Domain shift
Online evaluation
Zero-shot
X
X
A.2 LIMITATION
In our ablation study (Table 26), we discussed why we use the outputs of the frozen model as pseudo-labels. However, in practice, using the outputs of the noise detector as pseudo-labels can perform better than the frozen model on most ID datasets. Due to cumulative errors in the noise model's 's outputs, performance can significantly drop few ID datasets. Therefore, we chose the frozen model's outputs for a more balanced performance. In future work, we aim to explore how to use the noise detector's outputs as pseudo-labels while ensuring they work well on all datasets, thus achieving stronger performance the 7S-NTTA task.
Moreover, we utilize the detection results from ZS-CLIP as pseudo labels because CLIP's zero-shot OOD detection capabilities have been thoroughly investigated (Ming et al., 2022; Wang et al., 2023; Jiang al., 2024; Esmaeilpour et al., 2022) and have demonstrated exceptional accuracy across diverse ID/OOD datasets. However, our method may also falter in scenarios where zero-shot CLIP's detection accuracy is significantly low. Under such cireumstances, all existing zero-shot OOD detection methods would also fail. To address this, We may leverage the target data to fine-tune the model, potentially achieving better classification and detection accuracy.
16

---

Published as a conference paper at ICLR 2025
B RELATED WORK
Test-time Adaptation. fest-time adaptation (TTA) (Wang et al., 2021; Liang et al., 2023; Niu et al., 2022; Fleuret et al., 2021; Boudiaf al., 2022; Prabhudesai et al., 2023; Lee et al., 2024; Gui et al., 2024) aims to bolster a model's generalization to the target distribution. Given the unavailability of source distribution data in the test phase, various TTA methods have been proposed. Some methods (Wang et al., 2021; Niu et al., 2022; Fleuret et al., 2021) leverage self-supervised strategies like entropy minimization, while others employ techniques such as batchnorm statistics adaptation (Schneider et al., 2020; Nado et al., 2020) to improve performance on the target distribution. Some works (Shu et al., 2022: Feng et al., 2023; Karmanov et al., 2024; Samadh et al.. 2023; Ma et al., 2023; Zhao et al., 2024; Yoon et al., 2024) tackle the TTA problem with VLMs. TPT (Shu et al., 2022) and DifFTPT (Feng et al., 2023) learn adaptive text prompts with single test sample employing entropy minimization. TDA (Karmanov al., 2024) uses training- traiiing-free dynamic adapter to enable efficient TTA in iisoonlanguage models. However, they did not consider how to handle the presence of noisy samples in the data stream. In this work, consider the possibility of noisy data streams during the TTA process and cover the clean data stream case.
Noisy Test-time Adaptation. Recent works have considered noisy scenarios during the TTA process, and their emphasis has been solely on task-specific models utilizing visual data exclusively. Specifically, SoTTA (Gong et al., 2023) proposed using high-confidence samples to update the model, but they did not consider detecting noisy samples and only focused on the classification accuracy of ID samples. OWTTT (Li et al., 2023b) developed an adaptive threshold strategy for noisy TTA, but OWTTT relies on source domain prototype clustering, which is unavailable for VLMs like CLIP. Lee al. (2023) proposed utilizing the confidence difference between the original and adaptation models, but Lee et al. (2023) considered the long-term adaptation scenario, and this strategy may not effectively filter out the desired samples in the short-term adaptation scenario. Differing from these works, we introduce the zero-shot noisy TTA setting, which is more practical by leveraging the zero-shot capability of pre-trained VLMs.
OOD Detection. Different from the TTA setting, OOD detection (Hendrycks & Gimpel, 2017:
Yang et al., 2022; 2021; Fang al., 2022; Du et al., 2022: Huang & Li, 2021; Hendrycks et al., 2022:
2019b; Sehwag et al., 2021) 1) focuses on data with different label spaces. The goal is to detect OOD samples that are outside the label space of the training set. Most OOD detection methods (Hendrycks & Gimpel, 2017: Hendrycks et al., 2019a; Liu et al., 2020; Ming et al., 2022; Jiang et al., 2024; Esmaeilpour et al., 2022) design score function based on the confidence of the model's 's output, implementing detection in post-hoe manner. While SAL (Du et al., 2024) also leverages unlabeled test data to train robust OOD classifiers, our work differs in its focus and contribution. We primarily address the ZS-NTTA task, where our core contribution lies in proposing a conceptual framework that decouples the detector from the classifier. This decoupling prevents classifier degradation during noisy sample adaptation, with pseudo-label-based detector training serving merely as one implementation detail of our approach. Recent work (Ming et al., 2022; Jiang et al., 2024; Wang et al., 2023; Cao et al., 2024) explores zero-shot OOD detection by leveraging tre-tr VL.Ms. MCM (Ming et al., 2022) constructs the classifier using ID class names and uses the maximum predicted softmax value between image and text features the OOD score. CLIPN (Wang et al., 2023) and NegLabel (Jiang et al., 2024) enhance detection performance by mining negative information. EOE (Cao et al., 2024) leverages LLMs' embedded expert knowledge to envision outlier exposure without requiring actual OOD data. Unlike the zero-shot OOD detection setting, ZS-NTTA requires noisy samples to be detected online. What's more, existing OOD detection methods focus more on detecting OOD samples and do not consider how to improve the classification accuracy of ID samples.
Pre-trained Vision-Languaal Models. Pre-trained lanian-language models such as CLIP (Radford et al., 2021), ALIGN (Jia et al., 2021), and Group (Xu et al., 2022) typically comprise an image encoder and a text encoder. They are trained on hundred-million- self-supervised contrastive learning (Chen et al., 2020). In the testing phase, VLMs encode input images and texts into embedding vectors and then carry out classification by comparing the similarity between image and text features. VLMs demonstrate excellent generalization capabilities due to the broad coverage of the training data distribution and the robust feature representations learned through
image-text pair data using
17

---

Published as a conference paper at ICLR 2025
contrastive learning. They have also been effectively applied to downstream tasks like image retrieval and image classification in a zero-shot manner.
C ADAPTIVE THRESHOLD VS. FIXED THRESHOLD
To verify the reliability of the adaptive threshold used in our experiments, we compared the perfor- mance of the adaptive threshold with fixed thresholds across various ID datasets, where the fixed threshold ranges from 0.1 to 0.9. Due to time and computational resource limitations, we conduct experiments on the following ID datasets: CIFAR-10, CIFAR-100, CUB-200-222
STANFORD CARS, Food-101 101, and Oxford-III IIIT Pet. All ID datasets are tested on their respective four OOD datasets, and the specific ID-OOD dataset correspondences can be found in Table D. The average metrics are shown in Fig. It is clear that in terms of Accn, the adaptive threshold consistently surpassed all fixed thresholds in ZS-CLIP, SoTTA, and our AdaND. The average performance of Tent and TPT using adaptive thresholds is comparable to that of the optimal fixed thresholds. We suppose this is because the classifiers of Tent and TPT experience performance degradation due to noisy samples. Since adaptive thresholds do not require hyperparameter tuning for different ID datasets, we employ the adaptive threshold strategy across all methods.
300
60-
AEG. Ve fard
40-
ACCs fard
XEC, teradogenn
ataphee
20
6 Danasets age Score Threebedd
610 Datasels Auerae 0O0 Scare Thredhoid
ZS-CLIP
(b) Tent
3.00
AccLEwyy 192
68
AEC, for fann
fortent
ADC en
ctuppiie stppiis
ACC, be akppiec
MCG for alappi
staplle
6 Dacscets Auerage 0op Scare Th
613 Datscets Karaae 000 Scare Threchol))
(c) SoTTA
(d) TPT
200
Accuecy
ACC, bost
adippie ADC for adapple
(c) Ours
Figure 6: Results about Adaptive threshold line) and fixed threshold(solid line) range from 0.1 to 0.9. Best viewed with zoom-in.
D EXPERIMENTAL DETAILS
D.I DATASET DETAILS
The division between ID and OOD datasets in the ZS-NTTA benchmarks is shown in Table 8. Note that the label spaces of the ID and OOD datasets do not overlap. We also report the ratio of class numbers between noisy and clean datasets in Table 9. To avoid label space overlap between ID and OOD datasets, the iNaturalist, SUN, and Places datasets used our experiments are subsets constructed by Huang& & Li (2021).
18

---

Published as a conference paper at ICLR 2025
Table 8: ID/OOD Dataset Division. v indicates an ID-OOD pair, while indicates it is not.
ID
iNaturalist
SUN
Texture
Places
SVHN
LSUN
CIFAR-10
X
CIFAR-100
CUB 200-2011
V
STANFORD-CARS
V
Food
Oxford-IIT Pet
V
ImageNet
ImageNet-K
ImageNet-A
ImaggeNetV22
ImageNet.RR R
Table Number of classes ID and OOD datasets. Each row shows an ID-OOD dataset pair with their respective number of classes.
CIFAR-I0
X
10:47
10:50
10:10
10:10
CIFAR-100
100:10
100:1
CUB-200-2011
200:110
200:50
200:47
200:50
X
STANFORD CARS
196:110
196:50
196:47
196:50
X
Food-101
101:50
101:50)
Oxford-IUTT
37:110
37:50
37:47
37.50
ImageNet
1000:110
1000:50
1000:47
1000:50
ImageNai-K K
1000:110
1000:50
1000:47
1000:50
InageNNetA
200:110
200:50
200-47
200:50
ImageNet-V2
1000:110
1000:50
1000:47
1000:50
ImageNet-R
200:110
200:50
200:47
200:50
D.2 IMPLEMENTATION DETAILS
For the ZS-NTTA task, we integrated the advanced OOD detection method, i.e., MCM (Ming et al., 2022), into each comparative approach filter out noisy samples. For the Tent and TPT methods, all hyperparameter settings are kept consistent with their original papers. And we use the layer normalization in Tent when the image encoder is ViT-B/16 or ViT-L/14. For the SoTTA method, considering the generalization across ID datasets and based on the performance of different thresholds the memory bank, we set the confidence level of the memory bank to 0.5. For the ZS-OOD detection task, directly used the results reported in MCM, CLIPN, and NegLabel. Max-Logit and Energy are implemented by ourselves based on CLIP backbone. Additionally, to clearly illustrate our method, present AdaND in Algorithm 1.
D.3 ENVIRONMENT
The experiments presented this paper are conducted utilizing PyTorch 1.13 (Paszke et al., 2019) and Python 3. 10.8 within an Ubuntu 22.04 LTS environment, running on NVIDIA A100 80GB PCIc GPUs and AMD EPYC 7H12 CPU.
E ADDITIONAL RESULTS
We report the main results of CIFAR-10 and CIFAR-100 in Table 10 due to the space limitation in the main text. Compared to other methods, our AdaND achieves the best performance these two datasets. When using layer normalization, Tent supports bs 1. We conducted experiments with Tent (bs- 1) Table 11 and found that performs well on clean data streams. However, its performance degrades significantly when dealing with noisy data streams.
conduct additional experiments on more complex datasets, with results shown in Table 12. The results demonstrate that our method can outperform all the baseline methods. What's more, our method achieves the best ID classification accuracy Accş among all approaches. Note that ZS-NTTA inherently more challenging than traditional OOD detection, it requires simultaneous classifica- tion and detection capabilities under the noisy data stream. Specifically, ZS-NTTA requires online, real-time classification and detection results: for each input sample, the model must immediately
19

---

Published as a conference paper at ICLR 2025
Algorithm AdaND for ZS-NTTA and ZS-OOD detection tasks.
Require: test data stream {æ; [z ID class names Vis text encoder T. image encoder I, noise detector queue Q with capacity L, K len()ia), temperature 0.01, .M 8.
1: for test-time E f1.... T do
2: Calculate cosine similarity scores: I(a,) (T(c)
3: (sx( - (zx) H17 (d2)
) 4 E Va
4: Calculate OOD score:
5:
((x,) f max; Σx 2037
6:
Calculate λzs.CLIP by Eq.
7:
((x;) if
Azs CLIP then
8:
y
poe =1
> Pscudo-label: clean sample.
9:
else
10:
1Pe0 = -1
b Pseudo-label: noisy sample.
logit = f(I(z;))
12:
Update queue Q:
Q Q (I(z;), {
14:
if IQ) = L then
Train noise detector f:
16:
Calculate loss L using standard CE loss, input data: Q Update / using C
18:
2++0 0
Empty queue Q.
i mod M then
Gaussian noise injection.
20:
(0,
Add noise sample queue Q:
22:
logit = f[[())
23:
Q - Q (I(g), logit -1}
24:
if len()) (Q) = then
25:
Train noise detector f: Calculate loss L using standard CE loss, input data: Q Update f using C
26:
27:
28:
Q 0
29:
Generate output:
30:
if i < N then
Stage 1: use ZLIP
output + arg maxi (ak(a//++
31:
Ex e'SRU if S(x.) > λZs-CLI else -1
32:
else
b Stage 2: use noise detector.
33:
5(x;) +- maxk Σn 5
Calculate AAdanD by Eq.
output +- arg max k/rr
35:
Σx yoo0 if 5(x;) > AAdaND else -1
return output
20

---

Published as a conference paper at ICLR 2025
Table 10: Zero-shot noisy TTA results for CIFAR-10/100 as the ID dataset. The bold indicates the best performance on each dataset.
ID>
Method
SVHN
LSUN
Texture
Places
Avg
Apdy
Accu
Acts
Acci
Acts
Acdy
Acte
Acti
82.18
76.26
Tent
57.18
52.90
65.85
89.03
73.96
80.80
89. 78
88.48
89.13
88.78
65 44
75.34
88. 69)
70.19
77.78
CIFAR-10
SoTTA
90.21
81.71
85.75
90.13
91.06
901.59
89.56
90196
981.25
89.04
74.17
80.93
89.73
84.47
36.88
85.72
AdaND (Ouns)
94.39
58.56
89.60
98.54
93.04
91.31
97.79
93.34
ZS-CLIP
ZS CLIP
48.52
97.58
64.81
49.29
94.97
64,90
46.76
81.58
59.45
45.36
64.52
53.27
47.48
84.66
60.61
$5.39
42.41
60.06
83.37
69
59.31
57.52
62.24
59.79
66.79
61.36
CIFAR-100
SOTTA
60.56
89.24
72.15
60.28
88.89
58,
81.56
68.33
57.01
65.73
61.06
59.16
TPT
46.09
97.87
62.67
46.90
95.36
62. 8X
43.87
K3.10
57.42
42.48
66.86
51 95
44
85.80
58.73
AdaND (Ours)
64.44
99.78
78.31
62.42
99.15
76.61
65.17
84,84
73.72
63.50
14.21
52.13
63.88
81.99
70.19
Table 11: Performance of Tent with Layer Normalization (bs 1) on Clean and Noisy Data Streams
Clean Data Stream
Noisy Data Stream
ID
Method
Accş
Accy
Aceu
Aceg
Accy
ZS-CLIP
77.96
82.64
91.07
86.47
CIFAR-10
Tent (bs=1)
91.84
89.83
7.46
13.66
Tent (bs=64)
84.39
88.69
70.19
77.78
ZS-CLIP
44.69
60.61
CIFAR-100
Tent (bs=
63.66
37.86
20.64
19.82
Tent (hs=64)
41.90
58.07
66.79
61.36
determine whether it is ID/OOD, and ID, perform classification. In contrast, existing OOD detection methods typically report ID classification accuracy under the assumption a clean data stream.
Table 12: Zero-shot noisy TTA results more complex datasets. The ID dataset is ImageNet, and the OOD dataset is NINCO (Bitterwolf et al., 2023).
Method
Accs
Acex
Aсch
7S-CLIP
51.44
71.90
59.97
Tent
54.14
65.84
59.42
SoTTA
52.87
60.50
56.43
Ours (With Gaussian noise)
60.10
55.70
57.82
Ours (Without Gaussian noise)
50.25
77.99
61.12
We also compare approach to recent training faineng-free free TTA work, TDA (Karmanov et al., 2024), in Table 13. Our experimental results demonstrate that TDA's performance is inferior to ours. This indicates the necessity of training noise detector detect noisy samples in the Z.--TTAA setting.
Furthermore, our method is designed to be plug-and-play making it naturally compatible with existing TTA methods to enhance ID classifiers noisy data streams. To validate this compatibility, we integrate our method with Tent. Specifically, after updating the OOD detector for N steps (N = 10 in our implementation), the samples identified ID by the detector As shown in Table 14, the experimental results demonstrate that our method not only improves OOD detection accuracy but also enhances the classifier's intrinsic classification capabilities on ID samples under noisy data streams. These results validate that our approach effectively enhances the robustness of existing TTA methods when dealing with noisy data streams.
utilized to update the classifier.
To comprehensively evaluate our method's performance, we conduct extensive experiments on widely- used corruption benchmarks, including ImageNet-C, CIFARIO-C and CIFAR 100-C (Hendrycks Dietterich, 2019). Due to computing resource limitations, evaluate our method using iNaturalist as the OOD dataset for ImageNet-C experiments, and SVHN as the OOD dataset for both CIFAR-10-C and CIFAR- 100-C experiments, with corruption severity set to level-1. The experimental results in Table 15 demonstrate that our method consistently outperforms existing approaches on these corrupted datasets.
21

---

Published as a conference paper at ICLR 2025
Table 13:
Performance comparison with TDA using ImageNet as ID dataset. Results are averaged across four OOD datasets: iNaturalist, SUN, Texture, and Places.
Method
Accş
Acex
AcCи
ZS-CLIP
53.38
82.38
64.77
TDA
53.47
82.37
64.84
Ours (With Gaussian noise)
62,24
88.67
73.09
Ours (Without Gaussian noise)
60.64
91.73
73.00
Table 14: Results of integrating AdaND (Ours) with Tent for enhanced classifiers' classification performance in noisy data streams. Results are averaged across four OOD datasets (SVHN, LSUN, Texture, and Places) with CIFAR-10 as the ID dataset.
Method
Aces
Acex
AcCи
7.S-CTIPP
82.64
91.07
86.47
Tent
88.69
70.19
77.78
Ours
89.32
97.79
93.34
Ours (with Tent)
93.61
94.30
93.79
F FULL RESULTS OF ABLATION STUDIES
F.1 ABLATION STUDIES ON THE MODULES OF ADAND
Table 16 presents the ablation study results each module in our method across different ID) datasets. Experiments show that the noise detector is effective across different ID datasets, and after injecting Gaussian noise, the noise detector also handle the clean stream case well. Additionally, injecting Gaussian noise does not result a performance drop for our method on noisy data streams.
F.2 ABLATION STUDIES ON THE INTENTIONALLY INJECTED NOISE
Table 17 presents the results for Gaussian, Uniform, Salt-and- Salpeand-pepper, and Poisson noise the injected noise types. The results demonstrate that all noise types effectively manage both clean and noisy data streams, suggesting that our method is robust different choices of injected noise. Table 18 presents the ablation results for varying frequencies of Gaussian noise injection. Our experiments indicate that injecting Gaussian noise every 2, 4, or 8 samples consistently produces strong performance.
F.3 ABLATION STUDIES SIMULATING REAL-WORLD ADAPTATION
Table 19 shows the results for the zero-shot noisy TTA task across data streams with varying noise ratios, All competing methods show significant performance degradation on clean data streams while AdaND effectively handles clean data streams. What's more, method consistently achieves strong results across different noise levels. The results of different orders are shown in Table 20 and Table 21. Experiments demonstrate that our method consistently achieves top performance, regardless of the data stream's 's input order.
F.4 ABLATION STUDIES ON HYPER-PARAMETE SELECTION
Ablation studies on varying queue capacities L, queue lengths Ng. and optimization steps N are presented in Table 22, Table 23, and Table 24, respectively. The results demonstrate that AdaND is robust changes in these syper-parameters. For the main experiments, we T, 128, N, = 512, and found that N 10 optimization steps sufficient to initialize the noise detector. Overall, AdaND shows low sensitivity to syper-paramedt choices, achieving optimal performance across all datasets. Table 25 explores the performance with different backbones. Our AdaND consistently outperforms other methods across backbone configurations.
Intuitively, once the noise detector outperforms ZS-CLIP in detection results, it would be more accurate to use its outputs as pseudo-labels. We conducted experiments with various noise ratios and ID datasets in Table 26. Although using the outputs of the noise detector pseudo-labels
22

---

Published as a conference paper at ICLR 2025
Table 15: Experiments on CIFAR-IO-C, CIFAR-100-C, and ImageNet-C. Results are averaged across 15 corruption types.
Method
Accs
Accy
Accj
Accs
Accy
Accı
Accs
Accy
Acca
ZS-CLIP
74.14
98.24
83.96
39.43
96.74
55.49
44.14
84.73
57.91
80.25
57.86
66.83
45.29
41.15
42.83
37.10
16.50
22.79
SoTTA
83.68
85.00
84.06
51.46
88.70
64.73
46.41
62.48
53.21
Ours
82.15
99.89
89.84
54,89
99.18
70.18
52.63
94.78
67.55
Table 16: Ablation studies for each module in the method. For CIFAR-10/100, results are averaged across four OOD datasets: SVHN, LSUN, Texture, and Places. For other ID datasets, averaging includes four OOD datasets: iNaturalist, SUN, Texture, and Places. 'x' indicates the exclusion of module and V indicates inclusion of a module.
Clean Data Stream
Noisy Data Stream
ID
Noise Detector
Gaussian Noise
Accş
Accy
Acca
Accg
Accy
77.96
82,64
91.07
86.47
82.07
83.19
90.39
86.40
CIFAR-I0
67.89
89.14
98.37
93.51
89.16
89.32
97.79
93.34
44.69
47.48
84.66
60.61
43.40
46.07
85.84
59.77
CIFAR-100
35.21
61.65
90,42
73.23
62.52
63.88
81.99
70.19
33.08
37.95
85.71
52.59
CUB-200-2011
36.74
40.34
82.82
54.22
30.01
50.42
93.49
65.51
49.47
52.10
39.02
$2.65
98.13
68.53
44.08
$3.69
97.81
69.32
STANFORD-CARS
34.80
62.59
76.89
58.53
62.80
99.66
77.05
72.93
80.62
94.65
87.07
Food-101
77.61
80.79
94.56
87.13
56.75
86.46
99,00
92.30
86.21
86.44
98.85
92.23
70.17
79.53
87.73
83.41
Oxford-IIIT Pet
78.41
80.47
86.39
83.31
62.95
85.54
98.27
91.47
84.91
85.84
98.06
91.54
47.68
$3.38
64.77
50.07
53.95
81.65
64.95
ImageNet
37.54
60.64
91.73
73.00
63.96
62.24
88.67
73.09
30.48
33.41
79.33
47.01
31.43
33.72
78.67
47.20
ImageNet-K
26.03
37.70
36.54
39.78
83.1.
53.77
31.47
34.22
77.81
47.53
ImageNet-A
26.13
39.39
90,47
54.87
45.20
43.36
81.06
56.41
43.12
47.39
81.47
59.99
ImageNet-V2
44.93
48.03
80.88
60.25
32.17
54.43
91.31
68.18
$7.34
61.60
86.31
71.81
60.70
62.93
84.95
72.19
ImageNet-R
47.20
70.25
94.52
80.57
71.54
71.13
92.28
80.25
can result in better performance on some datasets, it can also lead to severe performance drops in certain cases, which is intolerable. For example, using ImageNet the ID dataset with 50% noise ratio, the performance drops from 73.09% to 41.34% when using the outputs of the noise detector pseudo-labels. We suppose this discrepancy arises from cumulative errors when using the noise
23

---

Published as a conference paper at ICLR 2025
Table 17: Ablation studies for the different injected noise in the method. For CIFAR-10/100, results are averaged across four OOD datasets under the noisy data stream: SVHN, LSUN, Texture, and Places. For other ID datasets, averaging includes four OOD datasets under the noisy data stream: iNaturalist, SUN, Texture, and Places.
Noise Type
Cleun Data Stream
Noisy
Data Stream
Accy
Acey
Acea
Aces
Accy
Acca
Gaussion
89 14
89.32
97.79
93.34
CIFAR-10
Uniform
Salt-and- pepp-e
89.07
89.25
97.80
93.31
Poisson
89.07
89.28
97.90
93.37
Ganssian
62.70
63.88
99
70.19
CIFAR-100
Uniform
Salt-and-peperr pepper
62.43
62.79
64.24
64.48
80.70
82.92
71.25
69.25
Paisson
62.80
63.98
80.94
69.38
Gaussion
49.53
52.10
90.77
66.14
CUB-200-2011
Uniform
49.53
$2.09
90.95
66,19
pettaad--epee
Poisson
48.91
52.01
90.98
66.13
Gaussian
58.61
62,80
99.66
77.05
STANFORD
Uniform
77.07
Salt-and-papper
57.76
62,70
99.65
76.97
Poisson
58.44
62.79
99.67
77.04
Gaussian
86.23
86.44
98.85
92.23
Food-101 101
Uniform
86.26
86.46
98.86
92.24
Salt-and-pepper
Poisson
86.21
86.43
98.38
92.23
Gaussian
84.95
85.84
98.06
91.54
Uniform
Oxford-UIT
Salt-and-pepger
84.88
85.82
98.
91.58
Poisson
84.56
85,78
98.21
91.57
Gaussian
63.99
62.24 88.67
73.09
ImageNet
Uniform
Salf-and-pepger
64.34
64.63
62,58
62.42
88.21
88. 11
73.05
73.13
Poisson
64.20
62.30
88.41
73.03
Gaussian
36.43
39.78
83. 15
53.77
Uniform
ImageNot-K
Salt-and-pepper
37.28
40.20
82.
53.97
Poisson
36.92
40.10
82.48
53.92
43.36
ImageNet-A
Uniform
45 28
43.38
81.00 0.09
56.43
Salt-und pepper
44.24
42.75
82.47
56.24
Poisson
42.90
Gaussion
58 .9
$6.05
$5.89
67.72
ImageNes-Ve
Sall-and- pepper
Uniform
58.44 44
55.99
85.51
67.56
67.75
Paisson
$8.20
55.89
85.68
67.53
ImageNet-R R
Uniform
71 .44
71.14
92.32
80.27
Sall-and-pepper pepper
71.08
70.98
92.52
80.25
71.07
detector's results as pscudo-labels. To better handle varying ID datasets and noise ratios, ZS-CLIP's result as pseudo-labels, which is more robust.
G FULL RESULTS OF FAILURE CASE
Besides evaluating different TTA methods using the rank distribution Figure 2, we also evaluate them using the absolute accuracy in Figure 7. Most TTA methods perform worse than Z.SCLIPP under the ZS-NTTA setting, and our method still performs best.
G.1 THREE MODEL ADAPTATION PIPELINES
Table 27 and Table 28 presents the performance of the three model adaptation pipelines using different datasets as the ID. For comprehensive evaluation, also include AUROC and FPR95 metrics for different datasets in Table 29 and Table 30, Higher AUROC and lower FPR95 scores indicate superior performance. Note that AUROC and FPR95 cannot be calculated at test-time and can only be determined after evaluating all samples. The above results show that for most datasets, the model performance degrades as the number of noisy samples used for model updates increases.
24

---

Published as a conference paper at ICLR 2025
Table 18: Ablation studies for the ratio of Gaussian noise in the method. '-' indicates that Gaussian noise is inserted, while '8' means that 1 Gaussian noise sample is inserted for every 8 test samples. For CIFAR-10/100 results are averaged across four OOD datasets under the noisy data stream: SVHN, LSUN, Texture, and Places. For other ID datasets, averaging includes four OOD datasets under the noisy data stream: iNaturalist, SUN, Texture, and Places.
Clean Data Stream
Noisy Data Stream
ID
Ratio
Accy
Acex
AccH
Accş
Acex
Accy
2 89,48 .99
90.70
CIFAR-10
4 89.32
89.41
96.28
92.67
89.14 8
89.32
97.79
93.34
2
65.79
66.02
67.83
61.75
CIFAR-100
4 65.68
64.90
65.18
62.70 8
63.88
81.99
70.19
2
54.03
$4.29
74.27
62.30
CUB-200-2011
4
52.94
53.72
85.17
65.68
49.53 8
52.10
90.77
66.14
2
62.84
63.20
98.88
77.11
STANFORD-CARS
62.19
63.08
99.56
77.23
2
86.46
86.57
97.71
91.80
Food-101
4
86.34
86.55
98.60
92.17
86.23 8
86.44
98.85
92.23
2
85.71
86.00
94.01
89.80
Oxford-IITT Pet
4
85.30
85.94
97.52
91.36
84.95 8
85.84
98.06
91.54
2
65 80
63.91
81.72
71.58
ImageNet
4
65.43
63.14
86.00
72.73
63.99 8
62.24
88.67
73.09
2 43.06
42.41
72.62
$3.34
ImageNet-K
40.98
41.16
78.76
53.97
36.43 8
39.78
83.15
53.77
2 46.24
47.07
36.45
40.29
ImageNet-A
4
46.05
45.72
62.37
$2.20
45.31
43.36
81.06
56.41
2
$9.29
58.85
65.84
61.56
ImageNet-V2 V2
4
58.77
57.52
77.51
65.68
58.39 8
$6.05
85.89
67.72
2
73.28
72.41
85.17
ImageNet-R
4 72.94
71.83
89.61
79.55
71.52 8
71.13
92.28
80.25
74, 70%
Accurscy (S)
66.89%5
68%
55
(Ours)
Figure Average absolute accuracy for all methods across 44 ID-OOD dataset pairs. The red dashed line indicates the performance of ZS-CLIP.
G.2 SCORE DIFFERENCE
The score distributions for TPT under the Normal pipeline are shown in Figures 8. Since TPT resets the model after updating each sample, the impact of unfiltered noisy samples is limited to the current step and does not accumulate. Despite this, the score of some noisy samples may increase, while the score of some ID samples may decrease, leading to decline in performance.
25

---

Published as a conference paper at ICLR 2025
Table 19: Ablation studies for different noise ratios the data stream. For CIFAR-10/100, results are averaged across four OOD datasets: SVHN, LSUN, Texture, and Places. For other ID datasets, averaging includes four OOD datasets: iNaturalist, SUN, Texture, and Places. Note that 0% indicates the clean data stream. The bold indicates the best performance each noise ratio.
0%
25%
50%
75%
Method
Accs
Acey
Accui
Accş
Acce
Accy
Accs
Accy
Acc
Accy
Aocy
Acci
ZS-CLIP
77.96
81.83
91.82
86.39
82.64
91.07
86.47
83,29
90.27
86.42
Tent
84.39
88.62
89.07
88.66
88.69
70.19
71.78
80.99
30,33
42.93
CIFAR-I0
S6TTA
83.82
88.29
90.58
89.26
89.73
84.47
86.88
90.14
64.30
73.56
TPT
76.37
80.02
92.22
85.56
80.90
91.52
85.72
81.53
90.83
85.72
AdaND (Ours)
89.16
89.29
95.85
92.43
89.32
97.79
93.34
89.10
95.75
92.21
ZS-CLIP
44.69
46.35
85
59.77
60.61
83.65
61.43
Tont
53.54
56.77
82.97
67.19
58.07
66.
61.36
53.17
41.63
45.14
CIFAR-100
SoTTA
52.62
56.25
67.57
81.36
68.34
72.69
IPT
41.90
43.72
86,30
57.84
44.84
85.80
58.73
46,01
85,10
59.57
AdaND (Ours)
62.52
63.24
75.14
65.29
63.88
81.99
70.19
64.28
82.21
70.83
CLIP
33.08
35.65
50.72
$2.59
82.05
54.54
Tent
36.69
39.14
82.75
53.08
37.75
54.78
44
31.45
21.34
25.21
CUB-200-2011
SOTTA
36.16
55.
76.30
TPT
32.07
34.93
89.49
50.24
37.30
87.59
52.31
39.84
84,66
54,18
AdaND (Ours)
49.47
51.00
86.08
63.98
52.10
90.77
66.14
53.39
83,99
65.17
ZS-CLIP
39.02
47.44
98.84
64.10
52.65
98.13
68.53
54.82
97. 44
70.16
Tent
40.95
49.33
97.88
65.60
$1.83
85.22
641009
33.88
29. 29.26
30.72
STANFORD CARS
TPT
38.38
46.
99.02
62.98
51,70
98. 36
67.77
54.06
97. 79
69.63
AdaND (Ours)
58.53
62.41
99.03
76.57
62.80
77.05
63.10
99.75
77.30
ZS-CLIP
72.93
79.34
95.59
86.71
80.62
94.65
87.07
81.50
93.96
87.28
Tent
76.20
81.47
85.00
82.68
80.87
69.34
71.90
63.38
30.37
39.10
Food-100
75.02
36.96
TPT
71.92
78 78.49
95.79
86.28
79.77
95.00
86.73
80.64
94.33
86.95
AdaND (Ours)
86.21
86.36
98.31
86.44
92.23
98.53
92.12
7S-CLIP
70.17
77.99
89. 34
83.27
79.53
87.73
83.41
80.96
85.69
83.24
Tent
73.36
79.
86.64
83.10
80.84
70.91
75.41
74.81
32.87
45.64
Oxford-IITT
SOTTA
72.58
79.61
83,30
81.36
82.66
78.21
TPT
69.44
76.98
90.96
83.38
78. 76.56
89.78
83.78
79.95
87.75
83.66
AdaND (Ours)
84,91
85.39
90.80
85.84
98.06
91.54
97.59
75 CLIP
47.68
51.00
84.72
63.66
$3.38
82.38
64.77
55.64
79.64
65.50
Tent
49.86
53.18
78.58
63.43
53.67
64.05
$7.66
49.74
45.93
45.89
ImageNet
SoTTA
49.82
52.36
74.92
61.63
$3.39
67.16
59.47
52.53
$7.03
54,68
TPT
46.12
49.48
86.38
62.91
51.85
84.48
64.25
54.04
82.24
65.21
AdaND (Ours)
63.96
62.53
86.82
72.62
62.24
88.67
73.09
61.53
85.52
71.52
ZS-CLIP
30.48
31
81.26
45.83
33.41
79.33
47.01
34,76
77.17
47.93
Tent
33.66
35.40
71.84
47.37
35.07
58.91
43.11
31,60
41.60
34.34
ImageNet-K
SoTTA
34,20
35.70
72.08
47.74
36.23
65.32
46.60
35.44
56.08
43.43
TPT
28.78
30.15
83.59
44.30
31.50
81.95
45.50
32.73
$0.22
46.49
AdaND (Ours)
36.54
38.40
85.81
52.98
39.78
83.15
$3.77
40.02
78.07
52.91
7S-CLIP
31.47
32.94
79.21
46.52
34,22
77.81
47.53
35.67
75.87
48.52
Tent
32.09
33.55
78.14
46.94
34,70
75.81
47.60
35.38
70.22
47.05
ImageNet-A
SoTTA
33.43
34.72
78.06
48.06
36.25
75.79
49.04
38.23
71.33
49.77
TPT
30.45
32.05
80.92
45.91
33.37
79.558
47.02
34.87
78.02
48.20
AdaND (Ours)
45,20
42.84
70.45
52.86
43.36
81.06
56.41
44,06
73.46
55.00
7S CLIP
43.12
45.0
83.36
58.
47.39
81.47
59.92
49,
78, 78.94
60.64
43.46
58
77.33
59.25
65.09
ImageNet-V2
SoTTA
43.87
46.38
80.75
58.91
47.89
77.03
59.06
49.01
70.30
57.75
TPT
41.53
44.04
85,38
58.09
46.00
83.76
59.37
47.88
81.68
60.33
AdaND (Ours)
58.42
56.37
76.77
64.70
56.05
85.89
67.72
56.34
83,12
67.04
ZS CLIP
$7.34
59.92
87.67
71.11
61.60
86.31
71.81
63.14
84.79
72.30
72.68
72.30
65.95
ImageNet-R
SoTTA
61.62
64.81
85.56
73.67
66.50
81.05
72.99
68.14
70.
69.07
56.20
70.71
60.61
71.40
71.98
AdaND (Ours)
71.54
71.23
91.61
80.05
71.13
92.28
80.25
70.95
88.95
78.79
G.3 GRADIENT ANALYSIS
Figure 9 shows the impact of clean and noisy samples on the gradients in Tent. To present clear view, Figure 9 only displays the portion of the gradient magnitudes less than 0.0010.
26

---

Published as a conference paper at ICLR 2025
Table 20: Zero-shot noisy TTA results for LIFAR-10/100 as the ID datasets with different random seeds. The results are the mean + standard deviation with five random seeds. The bold indicates the best performance on each dataset.
I
Method
SVHN
LSUN
Texture
Plaaes
Avg
Aceg
Acty
Accy.
Aodg
Actyy
Acty
Act;
Addy
Actiy
Accy
Addy
Asis
Acciy
Acti
ZS-CLIP
K3.53
98.35
90.33
83.10
97.83
59.87
82. 20
91.83
86.75
51.73
76.46
79.00
K2.64
91.11
86.49
= 0.00
0.04
5 0000
= 0.00
= 0.01
0.00
0.04
= 0.01
= 0.02
0.02
0.12
1 0.05
8: 0.03
0.05
0.01
Tent
87.31
54.02
66.70
58.54
70.43
78.43
80 66
$5.6 67
16
65.65
64.85
74.90
88.54
60 49
77.
0.30
5.13
2,50
0.53
2.71
1.90
0.00
0.05
0.10
0.58
5 0.40
0.30
1.64
1.19
CIFAR-I0
SOTTA
80.96
50.08
54-72
90.14 14
91.76
69
80.5 51
90.94
90.22
R0
74.07
80.94
89.71
14.09
66.64
2.00
1.19
0.35
0.17
0.77
0..44
98.89
89.53
50 46
85.80
77. 30
91.58
R5.74
0.09
0.04
0.004
0.05
0.03
0.12
0.06
1.015
NT (Ouns)
99.87
94.32
93.64
89.63
93:43
91.49
97.91
93.33
0.16
0.004
0.10
0.54
0.19
=
0.05
0.55
2 0.25
0.24
0.221
0.20
28 CLIP
48.50
97.59
61.30
49.17
95.05
61.81
46.78
$1.63
59.08
45.37
51.41
53.25
47.16
B4.68
50.59
= 0.00
0.0.44
= 0.07
1 0.08
= 0.06
0.06
0.00
= 0.03
= 0.04
0.06
0.13
2 0.06
# 0.07
0.06
0.06
Tent
54.72
41.45
47.17
59.80
83.27
69.61
59.07
79:42
67.74
57.36
62.08
59.63
57.74
66.55
61.04
=: =.442
0.89
0.71
12 0.31
0.24
0.27
0.22
= 0.22
0.00
0.10
0.26
8 0.12
8 0.26
0.440
0.330
CIFAR 100
SOTTA
60.30
89.43
72.03
59.91
89.24
71.69
58.63
S1.70
68.27
56.92
65.76
62.02
58.94
K1.53
68.25
0.44
0.18
0.11
11225
1.112
TPT
97.88
62.56
46.69
02.70
43.92
83,30
42.47
66.71
X5.33
58.67
ZS NTTA (Ouns)
77.75
75.97
85.43
62.01
$4.80
R3.35
70.40
0.74
0.06
0.5
1.00
0.8
1.21
1,50
=
1.16
2.89
1.49
1.17
0.99
Table 21:
Zero-shot noisy TTA results for CIFAR-10/100 as the ID datasets with different random seeds in terms of AUROC and FPR95. The results are the mean + standard deviation with five random seeds. The bold indicates the best performance on each dataset.
SVHN
LSUN
Texture
Places
Avg
ID
Method
AUROCT
PPR95]
AUROCT
FPR95.
AUROCC
APR95]
AUROCT
PPR9S|
AUROCT
FPR95
ZS-CLIP
98.45
6.75
97.75
10.64
94.75
28.08
87.47
50.18
94.50
23.91
0.00
+: 0.80
+ 0.000
+ 0.00
0.00
0.000
0.00
+ 0.00
+ 0.000
+ 0.00
Tent
75.11
48.89
87.08
34.46
96.87
16.01
87.64
46.19
86.68
36.39
2.21
3.14
1.79
1:2.5
0.10
0.38
1.00
1: 1.55
CIFAR-I0
SoTTA
95.27
22.67
97.72
11.47
97.32
13:05
91.57
33.88
95.47
20.27
0.66
4: 2.38
4. 0.36
4. 0.73
0.03
0.18
0.09
4: 0.15
4: 0.3
4. 0.96
TPT
98.48
6.80
92.62
10.73
94.19
28:21
85,33
50.19
93.91
23.98
1 0.00
1. 0.02
1: 0.0.
1. 0.04
- 0.01
0.05
1. 0.03
1. 0.02
1.0.0
J. 0.03
ZS.N (Ouns)
99.95
0.13
99.82
0.41
0.58
98.80
2.38
99.57
0.87
4 0.01
A: 0.04
4: 0.00
4. 0.07
= 0.04
0.08
0.02
4: 0.08
4: 4.000
4: 0.07
ZS-CLIP
85.11
86.42
85.88
72.58
71.09
95.35
58.47
98.97
75.14
88.33
0.00
0.00
0.00
+ 0.00
0.00
0.000
0.00
0.00
* 0.00
Tent
45.44
81.05
84.67
62.67
80,38
73.38
68.94
91.61
69.
77.18
0.82
0,59
4: 0.33
0.76
0.08
-0.47
0.06
0.18
0.550
CIFAR-100
SOTTA
88.78
51.05
87.99
55.39
81.47
70.58
70.59
89.96
82:20
66.74
0.24
0.54
+ 0.33
+ 0.79
0.08
0.32
0:11
0.37
+ 0.344
# 00.1
TPT
84.81
86.46
85.39
72.59
55.61
98.97
73.86
#: 0.01
#: 0.06
#: 0.01
4: 0.00
= 0.02
0.00
4: 0.04
4: 0.00
# 0.00
#: 0.002
ZS-NITA (Ours)
99.06
3.76
98.23
5.95
93.11
21.45
77.72
67.59
24.69 92.03
0.13
0.95
+ 0.222
+ 1.32
= 0.63
2:37
0:75
+ 1.69
+ 0.43
* 1.58
Table 22: Ablation studies on the queue capacity L for noise detector updates in the method with CIFAR-10/100 as the ID datasets.
ID
L
SVHN
LSUN
Texture
Places
Avg
Aces
Accs
Accu
Accs
Acox
Acci
Accs
Asox
Acci
Aces
Acey
Accu:
Accs
Aces
Accn
32
88.92
99.95
94.11
87.98
99.77
93.50
89.09
97.86
93.27
88.95
87.98
88.46
88.73
96.39
92.33
64
89.37
99.96
94.37
88.21
99.68
93.59
89.57
98.39
93.77
89.49
90.16
89.82
89.16
97.05
92.89
CIFAR-10
128
89.46
99.90
94.39
88.56
99.66
93.78
89.60
98.54
93.86
89.65
93.04
91.31
89.32
97.79
93.34
256
89.03
99.75
94.09
88.01
99.08
93.22
89.21
97.69
93.26
89.16
93.68
91.36
88.85
97.55
92.98
512
88.36
99.49
93.60
87.14
98.34
92.40
88.29
95.91
91.94
88.21
90.78
89.48
88.00
96.13
91.86
32
61.31
99.91
75.99
59.80
99.65
74.75
60.52
86.47
71.20
17
57.55
57.34
59.69
85.90
69.82
64
63.94
99.80
77.94
61.56
99.48
76.06
63.63
85.98
73.14
60.06
50.86
55.08
62,30
84.03
70.55
CIFAR-100
128
64.44
99.78
78.31
62.42
99.15
76.61
65.17
84.84
73.72
63.50
44,21
52.13
63.88
81.99
70.19
256
64
99.38
77.59
61.10
97.82
75.22
64.17
83.45
72.55
63.65
41.41
50.18
63. 14
80.51
68.89
512
61.01
98.88
75.46
56.26
95.18
70.72
61.38
77.75
68.60
60.44
43.56
50.63
59.77
78.84
66.35
27

---

Published as a conference paper at ICLR 2025
Table 23: Ablation studies on the queue capacity N, for queue length to store the output score in the method with CIFAR-10/100 as the ID datasets.
SVIIN
LSUN
Texture
Places
Ng
Acos
Accx
AccHt
Acds
Acey
Acoh
Aces
Acox
Acca
Acds
Acoy
Accи
Accs
Acen
AccH
64
88.16
99.90
93.66
87.35
99.60
93.07
88,
98.81
93.48
94.59
91.76
88.33
98.22
92.99
128
89.41
99.90
94.36
88.61
99.63
93.80
89.63
98.52
93.86
89.65
93.09
91.34
89.32
97.78
93.34
CITAR-
256
89.46
99.90
94.39
88.53
99.65
93.76
89.62
98.52
93.86
89.65
93.08
91.33
89.31
97.79
93.33
512
89.46
99.90
94.39
88.56
99.66
93.78
89.60
98.54
93.86
89.65
93.04
91.31
89.32
97.79
93.34
1024
89.43
99.91
94.38
88.51
99.65
93.75
89.62
98.57
93.88
89.64
93.01
91.29
89.30
97.78
93.33
64
64.19
99.65
78.08
62.64
98.73
76.65
64.61
84.64
73.28
10
46.96
53.48
63.38
82.49
70.37
128
64.92
99.72
78.64
62.86
98.90
76.87
65,
84.35
73.60
63.51
46.15
53.46
64.14
82.28
70.64
CIFAR-100
99.76
78.48
98.97
76.73
65.33
84.55
73.71
63.58
44.45
52.32
64.06
81,93
70.31
512
641.44
99.78
78.31
62.42
99 15
76.61
65.17
84.84
73.72
63.50
44.21
52.13
63.88
81.99
70.19
1024
99.77
76.34
53.49
Table 24: Ablation studies for the different initialization steps in the method with CIFAR-10/100 as the ID datasets.
ID
Step
SVIIN
LSUN
Texture
Places
Avg
Acos
Acox
Acou
Accs
Acex
AcoH
Accs
Acex
Aooi
Accs
Addx
Accs
Acos
Accx
Acda
88.49
98.64 64
93.29
86.
97.99
92.05
89.27
97.48
93.19
89.51
92.45
90.96
88.52
96.64
92.37
CIFAR- 10
20
89.13
99.76
94.15
88.35
99.66
93.66
89.20
98.28
93.52
89.20
92.18
90.67
88.97
97.47
93.00
97.91
89.97
40
88.48
99.56
93.69
87.70
99.42
93.19
83.37
97.44
92.68
88.34 34
89.85
89.09
88.
96.57
92.16
87.49
63.39
98. 72
77.21
60.72
98.15
75.03
64.42
83.91
72,88
62.12
43.74
$1.33
62.66
81.13
69.11
CIFAR-100
20
63.75
99.61
71.74
62.76
99.00
76.82
64.14
85.99
73.47
62.68
45,20
$2.52
63.33
82.45
70.14
30
62.51
52
76.79
61.89
98.97
62,80
86.48
72.76
47.20 26
53.35
62.11
83.06
69.77
40
61 30
99.41
75.84
60.77
98.72
75.23
61.42
86.51
71.84
59.79
48.74
53,70
60.82
83.34
69.15
50
60.37
99.29
75.09
59.90
98.46
74.49
60.25
86.08
70.89
58.47
51.19
54.59
59.75
83.75
68.76
Table 25: Ablation studies on VLM's architecture with CIFAR-10 as the ID datasets.
Backbone
Method
SVHN
LSUN
Texture
Places
Avg
Apdy
Act's
Acty
Acdz
Acti
Accy:
Aces
Acex
Actp
Asex
Accu
25- CLIP
51.73
99.84
68.15
49.90
97.47
66.01
50.09
93.91
65.33
47.75
71.13
57.14
49.87
90.59
64.16
Tent
16.81
47.90
24.89
20:31
63.46
30.77
29.19
54.21
37.95
24.82
33.72
28 59
22.78
.9.82
30.55
36.23
TPT
$1.02
99.87
67.54
48.90
97.52
65.14
49.01
94.13
64.46
46.20
72.34
56.39
48.78
90.97
63.38
AdaND (Ours)
67.05
99.69
S0.18
63.29
98.39
77.03
68.95
96.85
80.55
68.66
84.88
75.91
66.99
94.95
78.42
Tent
90.54
36.08
51.60
93.52
80.30
86.41
93.94
90.35
92.11
93.42
68.95
79
92.86
68, 92
77.37
VIT-L/14
SoTTA
93.90
70.96
80.83
94.14
91 80
92.96
94.02
90
92.11
93.74
71.44
81 08
93. 93.95
81
86.74
TPT
96.55
93.15
90.
90.87
76.
82.16
90.16
89.79
AdaND (Ouns)
94.77
97.15
94.50
99.67
97.02
94.87
98.65
96.72
94.86
89.95
92.34
94.75
96.98
95.81
OOD
Dannity
Density
-0.4
4:000
$000
3008
(a) ZS-CLIP
(b) TPT
(c) Score difference: TPT CS-CLIP
Figure 8: Failure case analysis of TPT (Shu et al., 2022) ZS-NTTA. (a) and (b) show the score distributions of ZS-CLIP and TPT, respectively. ID dataset: CIFAR-10; OOD dataset: SVHN.
28

---

Published as a conference paper at ICLR 2025
Table 26: Ablation studies for pseudo-labels generated by the noise detector under various noise ratios. Red indicates performance drop when using the outputs of the noise detector as pseudo-labels in terms of AcCH. For CIFAR-10/100 results averaged across four OOD datasets: SVHN, LSUN, Texture, and Places. For other ID datasets, averaging includes four OOD datasets: iNaturalist, SUN, Texture, and Places. Note that 0% indicates the clean data stream.
ID
Seeudo-label
25%
75%
AcCs
Aoeu
Acdy
Acdy
Aocs
Acci
Accy
Aody
Acci
Noise Detector
89.16
89.42
75.47
74.31
89.53
98.94
94.00
89.34
99.24
94.03
CIFAR-10
Model
89.16
89.29
92.43
89.32
93.34
95.75
Noise Detector
64.82
65,20
72.57
65.80
66.33
96.08
78.44
66.53
75.22
61.74
Frozen Model
62.52
63.24
75.14
65.29
63.88
81.99
70.19
64.28
82.21
70.83
CUB-200-2011
Noise Detector
Frozen Model
49.47
52.47
51.00
53.72
86.08
88.55
63,98
66.84
52.10
53.94
90.77
95.85
69.03
66. 14
54.67
53.39
83.99
97.93
65.17
70.17
STANFORD CARS
Nuise Detector
62.07
62,82
99.27
76,94
63.11
99.66
77.28
63.37
99.75
77.51
58.53
76.57
77.05
99.75
Noise Detector
86.23
86.38
98.00
91.82
86.45
99.17
92.37
86.49
99.58
92.57
Frozen Model
86.21
86.36
98.31
91.95
86.44
98.85
92.23
86.51
98.53
92.12
Oxford-IIIT Pet
Noise Detector
84.95
90.77
98.81
Frozen Model
84.91
85,39
96.94
90.80
85.84
98.06
91.54
85,89
97.59
91.36
ImageNet
Noise Detector 66.23
66.15
26.11
23.30
65.57
48.47
41.34
65.21
47.62
40.08
Frozen Model
63.96
62.53
86.82
72.62
62.24
88.67
73.09
85.52
71.52
ImageNet-K
Noise Detector
45.42
45.72
30.26
24.83
45.61
98
62.30
45.55
99.13
62.42
Frozen Model
36.54
38.40
85.81
52.98
39.78
83.15
53.77
40.02
78.07
52.91
ImageNet-A
Noise Detector
45.52
45.06
20.25
26.84
45,49
$7.15
45.39
45.25
53.84
Frozen Model
45.20
42.84
70.45
52.86
43.36
81.06
56.41
44.06
73.46
55.00
ImageNet-V2
Noise Detector 58.59
58.67
19.27
27.53
57.98
50.21
45.42
57.31
51.58
44.22
Frozen Model
58.42
56.37
76.77
64.70
56.05
85.89
67.72
56.34
83,12
67.04
ImageNet-RR
Noise Detector 73.43
73.55
28.53
27.97
72.97
97.75
83.56
72.56
98.50
83.57
71.54
71.23
71.13
80.25
Table 27: Failure case study of existing TTA methods with CIFAR- 100 as ID dataset. Green indicates an improvement over Z.-CLIP in average AccH, while red indicates the opposite.
SVHN
LSUN
Texture
Places
Method
Azcs
Asce
Accy
Accs
Acee
Accs
Acdy
Accs
Acce
Acci
Aces
Accy
Accyi
2S-CL.PP
48.52
97.58
64.81
49.29
94.97
64.90
46.
81.58
59.45
45 15.36
64.52
53.27
47. 47.48
84 66
60.61
Text (CTT)
62.11
92.92
74.45
61.28
89.73
73.83
60:24
80.42
68.58
58.55
65.11
61.66
60.55
82.05
69:45 (-8.85%)
(Normal)
(+0.75%9
Tent (All-aplate)
52.41
29.85
54.74
59.92
57.21
58
75.83
57.98
61.08
59.49
56.01
56.67
(-5.35%))
SoTTA (GT)
61.28
94.23
74.26
60.64
91.56
72.96
59.37
81.91
68.54
$7.49
66.47
61.65
59.70
83.51
69.43
(+8.828)
SoTTA (Normal)
60.56
89.24
72.15
60.28
59
71.84
58.
81.56
68.33
57.01
65.73
61.06
59.16
36
68.34
(+7.74%)
SoTTA (All-update)
(47.81%)
TPT (GT)
51.07
98.11
69.72
51.77
95.52
69.62
52.32
82.85
61.14
51
67.13
58,20
53,09
85.98
65.41 (+1.819)
TPT (Normal)
46.09
97.87
62.67
46.90
95 36
62.88
43.87
85.10
$7.42
42.48
66.86
51.95
44.84
85.5 80
58.73 (-1.889)
TPT (All-update)
52.35
84.64
64.69
$3.84
87.67
66.71
51.01
62.39
56.13
49.87
39.74
44,23
$1.77
68.61
$7.94 (-2.67%)
29

---

Published as a conference paper at ICLR 2025
Table 28: Failure case study of existing TTA methods. Green indicates an improvement over ZS- C in average ACCH, while red indicates the opposite.
ID
Methed
Naairalii
SUN
Texture
Places
Avg
Alcc
Accn
Accs
Acco
Acco
Acoss
Acee
Accr
Accs
Aeccy
Acce
Accs
Accy
AcCy
25 CLIP
33.13
85.06
53.15
37.56
79.11
97.81
53.04
37.95
52.59
(GT)
81.67
57.02
58.13
43.19
30.96
57.02
57.80
43.23
57.32 (1..())7733))
(Normal)
46.95
41.40
45.56
34.98
41.27
74.83
52.08
37:75
8.21%)
(All-spcane
(-14.2)))
CUR-200-2011
16
86.33
56.65
57.58
42.45
82.75
88.45
57.40
42.43
$6.56
56.93 (4.35
(Nommal)
41.67
84.37
55.79
56.09
41.44
77.58
86.52
56.59
41.51
55.77 (13.199)
SoTTA (All updazet
84.24
55.78
41.30
77.12
86.49
56.42
41.71
53.66
55.64 (13.055)
TPT
90.78
48.48
48.22
82.99
90.42
03.15
48.42
59.80
(+1000)))
TPT (Normal)
57.41
89:57
52.76
37.49
89.67
52.67
36 88
$1.67
89.45
52.79
37.30
57.59
52.31 (.277))
IP spdare)
05.10
54.00 06.00
46 69
58.51
64.55
46.36
63.25
53,63
Z5 CLI Tent (GT)
50.25
95.59
06.11
53.28
98.81
69.2
53.49
99.09
09.05
53.21
98.05
69.00
52.56
98.14
68.45
52.14
95.00
67.35
55.22
98.21
70.09
55.42
98.25
70.87
55.16
97.45
70.05
54.48
97.23
09.3 (+ 3.55))
Tent (Nommal)
64.12
52.33
47.88
54.27
94.51
69.95
54.60
97.37
69.97
54.33
96.65
69:56
51.83
$5.22
64 09 (4 3690)
Tent (All-oplone)
41:25
40.75
41 00
42.71
54.01
47.00
19
10
33:10
35.85
64.96
66.
21
55.56
42.01
48.53
44.53 (-25 999)
STANFORD-CARS
SATTA SoTTA (Nommal)
51.51
52.20
05.86
92.84
67 59
66.25
54.81
55.05
98. 90.30
97.57
70.19
70.00
55.05
55.19
98.50
98.64
70.64
70.78
54.75
55.00
03
96-85
69.95
70
56,00
54.37
96,47
91.06
09 84 (+) 1999)
06-27
(10.8358)
SoTTA LALl updane)
51.32
92.79
06.09
54.75
97.06
70.16
55.05
98.50
70.05
54.70
96.78
69.90
53.96
96,43
29.20
(10) 7555)
TPT (GT)
58.00
97.52
72.85
00:08
99,00
74.75
19.81
99.26
74.04
59.81
95:50
74.43
-9.47
98.57
78, 18 615 7255
TPT (Normal)
09.24
96.97
05:31
52.40
08.83
69.69
52.75
99.37
68.39
52.42
98 29
06. 00.00
51.00
98.16
67.27
TPT (All-spdan))
55.00
81:33
66.25
58.08
89.74
70.50
59:00
16
72.90
53.13
90.14
70.66
57.98
89.14
70.09 (+)
ZS-CLIP
80.63
94.79
87.60
80:50
93.10
94.99
87.07
50.62
94.63
87.06
91.89
58.00
90.16
91.95
87.43
63:31
91,59
57
385)
(Nummal)
75.81
13.00
81.18
59:87
06.14
71,90
(All-opaate)
57.85
61.57
(-92-018)
Food-101
03.22
60.55
08:17
(40 519)
81.
$4.09
82.05
90.10
96.79
$0.30
SolTA (All--ppane
81.
82.26
87.41
81 11.99
59.35
82.34
91.25
95.57
82.00
85.0
55
$5.01--16699
TPT
84.50 14.55
95,11
99:41
$4.42
96.24
89 04
84.32
93.55
58.70
84.43
95.0 03.01
89.61
54.36
04:38
50.33 (+2.3110
TPT (Nomma))
70:70
94.91
$6.65
70.00
96.19
87 10
79
70
95.86
56.70
79:76
95. 14
86.77
79.77
05.00
86.75-0 399)
TPT (All-qpobe)
83.00
71.41
07.03
63.79
60.42
67.00
83.84 84
81.36
$2.58
83.95
78.85
11.52
$3.90
78.01
80.75 319)
25. CLIP
75,55
85.31
83.16
79.77
87.26
K3.35
80.12
91.17
$5.29
79.56
H4,30
81.86
79.51
57.76
83.42
Test (OT)
81.15
85
85.75
82.10 16
06.05
84.06
82.35
$0.00
$6.02
62.01
83.65
82:73
$1.02
$6.40
64 2258)
Test (Nommal)
80007
78:09
79007
$1.19
68.3
79.19
81 4k
74.72
77.95
80.64
62.51
70
50:54
70.01
75.41
(-8 019)
Test (All-oplane)
77.55
70.76
74.01
70.32
62.61
69.98
78.00
61.46
68.08
70.00
54.96
64.83
62.45
60.45
(-1 (13978)
Oxtord HIT Pet
SoTTA (Nonnal)
SoTTA (GT)
80.72
80007
85.37
83.54
$3.45
41.77
81.78
$2.00
83.53
89.37
62:29
18.18
82.09
82:51
87.52
90.42
54.72
56.28
81.70
81.49
H1.25
83.67
82.
81.37
$1.78
$1.36
$4.00
54.13
52.66
(4.13:40.77 728)
(0.75%)
SoTTA (All-
79.96
03.52
41.70
$1.55
83,63
62:58
$1.97
87.64
$4.71
81.57
81. 15
81.32
$1:21
$4,02
82.5%
(8855)
(GT)
83.
89.99 90
56
83.00
86
83.92
92:11
67.86
83.81
85 41
84
83.98
1900
16
(Nomaa)
77.56
89.71
78.57
89.82
83 09
17
92.26
87. 75.60
82 74
78-56
19.33
83.78
(All-spaac)
81.77
58.09
68.27
$3.43
62.39
71.39
83:26
70.69
76.46
83.18
59.06
60.06
53.15
62.56
71.30
25 CLIP
54.01
96.46
53.32
19
52.66
78.69
63.10
53.25
90,40
64.07
53.31
$2.35
56.15
79.09
55.93
78.31
65 25
55.34
72.69
55.81
75.31
61.11
55.81
76.45
(Nummal)
57
(779))
(All-ppaac)
45.05
31.25
54.25
52.46
61.00
55
3055
InageNic
55.51
75.20
63.5
54.91
73.63
63.15
74.38
65
(Nomaal)
53,15
62.65
53.16
53.6
69.16
60.39
53.39
67.36
59.47
(.5 (55)
(All updaze)
53.05
61.97
52.89
53.5
66.30
68.06
59.59
53.14
66.13
58.1
IPT
88.25
85.44
71.75
26
$0.45
82.53
70.43
61.64
$4.12
71.13
TPT (Nommll)
52.55
88.93
66
51.91
96.09
64.77
$1.11
$0.01
62.3%
$1.80
K2.80
63.76
$1.85
84.45
64.25
(-0-6))
TPT (All-apaac)
60.85
61.41
61.13
60.97
62,85
63.90
60:35
$7.91
59.30
60.70
61.99
61.34
60.71
61.04
60:57
(-3.855)
25 CLI
34,14
83.35
45.41
33.32
81.16
47.24
32.66
75.55
45.00
33.57
77.12
46.55
13.37
79.29
46.97
Tent (GT)
37.00
75.95
50.15
39.14
75.43
09.77
36
39
08.41
47.51
57.00
72.19
48.99
17.00
73.00
49. 10.10 (+2 199)
Tent (Nommal)
30.40
26.96
78.55
36.57
71.82
6% 46
16.33
66.65
47.06
5687
70.33
48 36
1807
58.1
4511
(-5
Teat (All-)
31.15
28.86
29:95
35.38
09.67
46:93
35.94
65.09
46.31
3500
69.07
45.35
24/62
56.17
42.63 (4.34))
InggeNe-K
SOTTA (GT)
37.69
72.29
49:55
37.60
75.21
50.14
36.93
70.68
48.51
37.51
71.81
49.25
27/43
72.10
(12 800%) 49.37
SoTTA (Nommal)
36.15
61.70
45.61
36.28
67.19
47.12
35.91
65.31
46.34
30.57
07.00
47:34
26.23
50:72
46
0.368)
SoTTA (AUl updane)
35.00 09.09
59.70
46.55
36.29
66.56
60:97
35.95
65.72
45.97
30.33
66.50
41.05
26.03
54.13
42 0.558)
TPT (GT)
39.52
85.67
54.29
19.34
51.89
51.56
36.95
76.30
52.00
39.21
80.83
52.23
19.20
82:32
55.15 (+b
TPT (Normal)
32,11
85.52
46.89
31.55
83.86
45.55 85
10 94
71.39
44000
51.56
80.05
45:21
11.50
$1,95
45 -0
(All-ppdaae)
35.25
59.33
46.51
38.45
60.41
46.99
37.96
$4.98
35,33
39.67
46.68
39.25
58,60
46.27 (-069))
CLP
$0.09
45:55
47.00
13.97
76.60
25.11
46.77
14:22
77.11
49.05
14.99
76.73
48.06
23.87
14.70
(All-apaae))
46.50
73.75
.77 (-111)))-11)
ImageNes-A
SoTTA
SoTTA (Nomma))
35,85
75.83
78.79
50.41
49.1
09.51
8..88
35 60
36
75.37
73.87
48.85
48.27
36.61
56.25
15:39
49,04
(12 00%5)
SoTTA (All-opdane)
35.87
75.91
49.85
36.55
69.57
35
75.08
36.30
23.79
48.72
56.40
15:0
49.16
(.9)
TPT (GT)
82.39
58.52
44.00
80.80
$7.47
44.67
79.19
57.12
44.55
77.51
56 36.00
44:77
70.50
$7,40
(+0 888)
TPT (Nomma))
54 12
81.17
46.04
$3.30
80.21
46.
13.12
79:42
46.85
33.05
77.00
46.25
55:37
70.58
47
(-0 505R)
TPT (All-apebnc)
43.31
55.05
46.86
43.05
$3.91
47.83
43.68
$8.71
50.09
42.99
$2.81
47.40
43.76
$5.38
48.56
039)
2S-ULPP
45.05
85.
77
61 59
47.43
61.53
60:45
46 72
77.70
58.15
47.45
79.44
5944
47.41
$1.56
50.95
Test (GT)
48.00 85.89
N2:71
61.45
48. 16
W0.94
60.30
47
75.31
58.99
14
77.73
59
48.18
79.17
59 90 4 (69))
Test (Numaal)
47.04
76.98
09.08
48-78
50.50
60.30
47.55
74.47
58.05
45.54
77.37
50
30.50
4800
77:33
59 (-0 700%))
Test (All-apaaa)
47.51
73.
57.59
47.52
70.52
50.40
47.47
73.03
57.82
47.87
76.55
58.90
47.59
75.78
58.45 e- 500%)
ImageNer V2
SeTTA (Nomma))
SoTTA (GT)
48 14.24
45.90
H2.74
78.50
61.39
59.76
47.50
48.23
80.61
78.87
60:35
59.67
47.63
47.27
74.82
76.11
57.9
58.59
45.22
65.25
77.03
16:05
19.05
59.31
48.22
47:89
1200
77.12
39
59901
1-0 (465)
(All-opdnn)
76. 24
59.00
47.51
14.64
59
47
74.42
57.98
15.10
75.0 57997
56.00
90
47.54
70.4
58
35.99 -0
(CT)
87:80
68.05
55.97
85.12
67
95
79.09
R1
66
10
55.92
56,50
(+5
(Nommal)
46.63
65.05
46.12
59.04
45.2
79.14
57.55
81.05
5K 94
46.00
83.76
59 37 58%)
TPT (All-apaacc
$4.50
60.62
$7.40
54.65
62.15
55.16
54.00
$6.35
55.15
54.46
61.45
57.76
$4,40
60.15
57.12
839)
7S-CLJP
77.97
6092
71.43
R6.33
65.45
74.90
64.63
$4.30
13.45 659)
Teat (Numnal)
65.22
01.45
73.05
63.35
82.
72.62
64.64
$2.36
12.30 (40.499)
(All-plaac)
64.65
90.75
72.47
62.22
81
80.56
71.12 (-0.99))
ImageNerr R
91.75
76.00
66.9
K2.49
74
67.42
84.36
14.87 (13.068)
(Nommat)
66.75
96.95
74.56
6592
72.69
50.53
72
66.30
$1.05
72.99 (11.19%)
SoTTA (All-opann))
66.61
85.68
74 96
66:00
81.64
74
65
71.65
6867
8006
73.73
66.52
66336
12.68 (408796)
TPT (GT)
70.30
05.01
80.7
76.34
8187
76.85
6981
72.91
7364
10.44
85
71.23
70.17
K3.14
17.55 (45884))
TPT (Nommal)
60.95
04.30
74.20
60.35
89.98
72.60
59.9%
77.70
67.73
606
85.79
71.08
4061
80.00
71 (-0-440)
TPT (All-)
69.10
72.12
70.58
69.14
66.38
67.73
68.64
$6.45
61.5
63.85
63.42
66
68.93
64.59
66.37 (-5.246)
30

---

Published as a conference paper at ICLR 2025
Table 29: Failure case study of existing TTA methods with CIFAR-10/100 as ID datasets. Green indicates an improvement over ZS-CLIP average AcCH, while red indicates the opposite.
Mathed
LSUN
Texture
AUROCY
FFR95
[PR951
JUROCC
PERS5
AUROC
FPR951
AUROC1
FPR95]
(GT)
1888)
(-9.338)
Tex: (Normal)
74.35
89:47
31.18
87
87.75
45.57
87
5018)
35.74 (+11
Tent uplltepdatel
02.78
05.12
73.23
$4.20
95.80
22:24
82.53
50.35
78.39 (-16022%)
49.48 <)25 5055
CIFAR-10
SoTTA (GT) SoTTA (Normal)
95.77
99.24
20.74
3.13
98.0 51
97.57
11.68
7.24
97.27
97.44
11.89
13.02
91.43
92.17
31.41
33.91
95.51 (-0.91%)
96.84 (-2 249)
13,42 (-142.-10.500 (-10.500))
19.84 (-4.08%)
SoT TA (Ail-update)
93.29
30.24
97. 46
12.79
97.21
13.76
91.47
33.75
54, (-0.25%)
22.63 (-1.2856)
TPT (CT)
99.28
3.07
98.0
4.61
95.99 04
91 31
35.75
96 59.59 (-1 98%)
14.55 (-9.34 (-9.34%%
IPT (Neemal)
10.67
50.18
(-0.99%)
23,90 (+0,055))
(All-update)
(-2.839))
75.14
58.33
(Nocmal)
(:1.335))
(Allupdate)
50.78
86-47
CIFAR-100
SoTTA (GT)
SOTTA TATTA (Normal)
92.29
53.72
41.42
51.10
59:60
87
51.31
$4.48
81.96
81.45
69.99
70.58
70.60
71.43
89.36
90.18
83,82
82.18 (-7 04%)
68%)
66.59 (-21.748)
63:00 (22.339))
SoTTA ull-updatte)
$3.99
49.96
87.35 77
55.49
$1.40
71.23
66
89.85
82.20 (-7 06%%
66263(-21.3 (-21.76)))
IPI (GT)
$5.66
76.97
89.25
63.17
76.87
900.7
66.
97.82
50.26 (-5. 12%)
82.13 (.6.2056)
TPT (Nommal)
$4.80
56.45
85. 37
72.58
09.62
95. 34
55. 59
98.99 97
73.8 (-1 29%)
88.35 (0.0005)
TPT (All-upuate)
75.97
94,94
82.55
$1.02
62.82
95.60
48.79
98.87
67.53 (-761)))
92.61 (+4.28%)
step
Total samples
Classified
Avg
cleun
gradicnt
stage
3259
2991
2.10-4
stage 2
2973
1.004
stage 3
3165
2697
1.4c-4
10,0010
0,0005
0,0000 gradient
0,0005
(a) Clean samples
step
Classifled
Avg gradient
samples
as clean
stuge
136
100
stuge
3219
1130
l.1e-
200
stage
3235
3060
1.0e-4
10,0110
.0.003
0.0000
0,0025
0,0010
grudient
(b) Noisy samples
Figure 9: The impact clean and noisy samples on the gradients. Note that the gradient magnitudes of clean and noisy samples are not the same scale; for clarity, the figure does not show gradients with magnitudes greater than 0.0010. The gradients of noisy samples are substantially larger in the first and second stages. The model effectively filters noisy samples in the first stage but gradually struggles to distinguish between clean and noisy samples. ID set: CIFAR-10: OOD set: SVHN.
31

---

Published as a conference paper at ICLR 2025
Table 30: Failure case study of existing TTA methods. Green indicates an improvement over ZS- in average AccH, while red indicates the opposite.
Mathod
SVHN
Testuee
Place
Avg
AUROCC
[9R055
|AUROC]
FPMS5__
AUROCC
|UROCC]
FPR951
AUROCT
75-CLIP
80.70
90.31
$0.18
61.71
09.44
79.84
62.83
Test
K1.24
$4.05
55.01
85.31
56.36
(43.5455)
58.24 (-5.09%)
Teat (Normal)
46.85
55.06
80.18
69.52
75.90)
52,99 (2254116)
82,59 (972227))
Teut update)
29.00
86.77
92.95
56.73
85.76
(3354005)
89.48
(626.166))
CUB-200-0011
SoTTA (GT)
SoTTA (Noemal)
$1.80
79.75
$2.00
$1.58
55.12
58.25
14.79
08.23
85.41
81:37
56.69
61
79.47 (+10076)
52.24 (+3.846)
57.45 (5.865))
62.69 (0.455)
SoTTAA updaze)
79.54
62.84
59.22
74.64
09:00
81.31
61 11.25
79.10 8916)
63,00
(0.255)
TPT (CTT)
90.36
41.25
90.37
43.41
53.11
90.05
45 13.84
88, (-10.756)
45.41 (-17.115))
TPT (Noemal)
80.52
59.54
41
73.67
26
79.97
62.60
78.57 (753
52.95 (-0.59))
TPT update)
72.08
71.40
79.71
70.77
(74396)
78.71
(135.291))
25-CLIP Test (CTT)
94,15
25.03
96. 18
8.36
98. 20
8.64
97.71
9.79
97.06
13:33
91.99 05.95
27.11 11
98.00 06
8.60
98. 10
8.91
97.67
9.80
96.98
(-01149)
13662 (-0, 1999
Test (Normal)
55,00
65.81
95.13
17.07
97. 10
13.56
96.75
13.39
$6.00
(11 (99)
27.46 (-14.089)
Test (All-ggsatc)
43.67
77.00
58.11
63.76
30.00
77.72
68.48
58.33
$2.49
(-44 5959
69.35 (455.92))
STANFORD CARS
SoTTA (CTT)
SoTTA (Nocmal)
91.62
94.47
34.005
26.115
99.10
97.31
11.47
8,43
98, 58,099
98.25
9.24
8.70
97.75
97.37
11.08
9.50
97.15
96.30
(40.075)
(-0.9955)
16.46 (-3.119)
13.25 (-0.159))
SoTTA (AUl updaze)
91.40
33,99
97.45
10.56
97.
9.90
97.24
11.59
96.01 (-1.085))
16.51 (-3,085E)
TPT (GT)
97.41
13.87
99.17
3.78
99.25
4.01
98.99
4,52
98,70 (+1.621))
6,54 (.899))
TPT (Noemal)
93.90
25.57
97.92
8.48
98,
5.84
97.66
9.52
96.97 0.1216)
13:35 (.085L)
TPT (All update)
87.21
45.63
95.42
22.78
90.10
12.61
95.80
21.41
92.88
(-1.006)
25.61
(+12.1816)
75-CT.P
97.71
11.35
98.10
10.16
96.52
13.09
97
13.05
97.48
11:91
Teat (GT)
07.57
13.26
98.64
11:55
96-44
14,66
97.59
13:00
97.41 (-0005))
13.34 (-1 (-1.438)
Teat (Noammal)
30.44
80.04
95.00
21:35
05,05
20.90
91.77
27.63
91.32
(-17.166))
37.48
(+25 (+25.575))
Test (ABl- update)
35.67
85.75
67.60
56.23
72.00
56,92
69:85
56.67
61.28 (36.205)
64.40
(.52.499)
Rod-100
SOTTA (Nocmal)
SOTTA (CT))
94.51
97.78
24.94
12,00
97.81
98.22
10.28
12.34
95.55 97
96.57
10.73
14.13
97.31
97.74
15.01
12.94
97.58
96.40
(+0.305))
1.08))
17.26 (-5.345)
12.54 (-0. (245)
SoTTA (All update)
93.96
27.45
97.21
13.12
05.78
17,53
97.17
15.76
96.10 L.331)
85 (-6.55%)
TPT (GT)
7.07
98.91
5.65
97.77
8.43
98 65
7.22
59.50
(+1 10225)
7.09 (-48226)
TPT (Noesall
97.18
13.73
97.84
10.42
96.09
13:24
97:35
13,07
97.21
(-0.278)
12.12 (-0.2008)
TPT (ABl update)
03.04
35,70
95.20
28.53
94.64
26.53
94.88
29.09
94.44 (.3.04%)
30.71
(+18,805)
2S-CLIP
94.15 16
50.75
21 68
96.64
16.79
93:48
26.51
94.64
23,94
Tent (GT)
94.90
98.91
21:39
16.87
94.07
26.07
12 (80-486)
23.31 (40668))
Test (Noemal)
00.23
86
81 56
50.01
96.70
(-70995)
1955
Test (Allugslate)
R6.45
30.66
50 18
80.20
65.50
75:25
58
50.64
(31000))
$4.95
Oxtoed HII Per
SUTTA (Nocmal)
SUTTA (CT)
94.44
03:13
93.64
94.78
24.75
20.98
95.79
96.71
19.91
16.27
95.85
92.52
94.04 (10.315)
93-77 (-0.755)
27.41 (-3.499)
23.24
700%)
SuTTA (ABl (All-update)
02.91
93.56
25.16
20.33
92.51
30.18
93 (009065)
27.96 (-4.0255)
TFT (GT)
97.60
12:25
98.59
8.25
97.19
97.50 (+3.169)
12,55 (-113399)
TPT (Normal
03.54
96.57
21.05
96.23
16.25
93.36
94.43
0.211%
23.19 (.755%)
TPT update)
$9.00
53.33
90.17
43.59
93.48
34.00
$6.39
47.78
90.35
4.09%)
(+20.7316)
7S-CT.PP IP
86.64
50.45
86
$2.98
58.31
83.71
59.91
80.57
62,80
(1100))
(-3.019)
(Normal)
79.70
64.76
(-12.09))
70.92
(412,609)
(All-ugdate)
77.6%
(-157226)
75.04 (417 (7.635)
ImageNet
SoTTA (GT)
SoTTA (Normal)
71.13
81.57
63.16
73.10
75.81
79.70
71.74
30.03
13.95 (-9000%)
(-2.05%)
73.51 (515.1915)
SoTTA update)
70.64
74.42
72.75
13.59
75.12
72.88
13.38
9.61%)
74.45 (136.1196)
TPT
92.46
43.32
80.3 35
(46.099)
(-13.589)
TPT (Noesall
$7.09
59.53
(-06225)
TPT upllttd
67.53
75.26
78.47
(-542%)
69.65 (+11,344))
25 CLIP Tent (GT)
75,13
78.00
71.38
83.15
66. 50
85.24
69.02
$2.68
20:46
$2,42
74.04
75,24
77.85
66.63
70.89
788
71.17 (+0.9154)
79, .9 (-2.99))
Test (Neemal)
28 86
93.52
69.55
81 79
64.5 S4
86.01
68.69
$2.37
$7.86 (-12,09))
85.90 91 (-1.519)
Test (All-gpustc)
$1.41
93.20
86.45
62.7% 7%
26
66.1
85.
56.83 (113625)
88.05 (-5.679))
ImageNat-K
SoTTA (CT))
SoTTA (Nonnal)
61.84
72.08
77.26
85.96
65.08
73.44
$4.
76.55
68 62.02
63.25
81.00 10
85.66 66
71.11
66.28
84. 24
77.
71.39 (40.939)
64.34 (-6. (-6.125)
84.99 (-2.579)
78.09 (-4.359))
SoTTA (All-update)
60.37
86.85
65.73
$4.49
63.59
16.17
65.95
$4.5
63/66 (6.5099)
N5.52 (-3.109)
TPT (GT)
K3.51
69:23
B1176
74.80
76.34
79.50
78:00
76.23
79.35 (49) (09.40%)
74.94 (-7.459)
TPT (Nocmal
74.55
75.10
71.52
$2.15
65.70
85.05
69
$2.05
70.24 (.0.22%))
$1.54 (-0.57%)
TPT (All update)
63.59
86.87
65.12
$5.90
60.34
90.42
64.26
$7.02
63.33
7.33%)
$7.55 (-5,129)
7S-CTJP
26
65.23
72.73
76 64
70.65
78. 17
70.04
78.07
22:41
75.70
Test (CT)
25
60.65
72.53
77.
70.41
70
70.08
78.47
12.17 (400489)
75.0 00 (-0. 7099)
Test (Noemall)
73.67
72.67
71.90
72.37
60:71
70.51
69.5%
79.07
11 (11.21%)
73.25 (-1 979)
Test (All-gddttc)
73.75
72.64
71.38
79.05
10.10
80.17
69:22
79 60
70.56 (11559))
77.87 (-2. 5848)
ImageNassA
SOTTA (Nocmal)
SOTTA (CT)
76.00
74.29
71.40
60:33
73,00
73.66
75.27
75.65
70.84
72.01
75.63
76.83
71:11
70:1
77.28
76
13.20 (40.789))
72.14 (.0.29%)
75.74 (-0.45%)
14.58 (0.719)
SoTTA (All update)
74.29
71.84
73,00
75.48
70.73
75.73
70-42
77.60
72.11 (0.30%
75.91 (-0.625%
TPT (CTT)
84.33
55.37
88
67.5
80:45
166.5 53
79.71
71 71.19
$1.50 (.0 1895
56.40 40 (-8 H8F6)
TPT (Noemal)
74.45
68.41
71.32
76.63
70.66
77.K5
6901
73.23
71.36 (-1.00%)
75.28 (-0.019)
TPT (All-uddate)
64.16
87.03
63.25
$6.79
66.39
83.79
61.6%
$9.65
63.57
(-8.54%)
86.81 (+11.536)
25 CLIP Tent (GT)
83.54
60.57
$0.49
67.14
72.67
78.36
69.03
79.36
67.35
K2.25
62.78
79.93
67.95
25.15 18
73.55
77.94
69.41
78.83
(0.7316)
42 (095)
Tent (Neemal)
77/24
65.87
19.65
67.56
74.65
73.91
77.75
60
77:45
(01))
20. 12 (-2 (655)
Tent (All-ggdate)
75 14/44
72.01
78.67
71:31
74
15.01
77.09
71.47
76.33
72.45
109)
SUITA (CT)
83.50
79.58
68.
79.04
77.80
70.07
78 58 (00485))
74 9099)
SUTTA (Noemal)
78.71
78.11
710 35
74.60
74.
76.99
72.12
77.17
1999)
71.20 (-4.
SuTTA (Allapshte)
76.72
69.80
75.15
70.67
76.96
72.
77.11 (-2.459)
71.05 (-4.609)
TPT (CT))
89:97
45.36
87.98
54.92
61.75
86
57.
67.19 639)
55.58 (-11.779)
TPT (Normaal
K2.67
50.43
66.19
75.24
72.63
78.39
68 48
79.18 (40.399)
56.86 499))
TFT update)
73.47
75.18
74.93
$1.09
74.53
75.83
73.53
(6.00%
76.72 (-9.38%)
CSP
(Noemal)
90.45
$7.50 59
(All-ugdatc)
62.67
(2917
InageNa-RR
SoTTA
91.56
50.67
8663
1995
54,07 (-4.66%)
SoTTA (Neemal)
$8.40
55.56
85.09
35.19
60.17 (-1.649)
(All-update)
87.64
50.54
84.95
(-10)
61.51
05.15
14
699))
(-18 (496)
TPT
90.54
$5.32 (40.66))
54.70
TPT (All-ugdate)
84.12
72.54
79.35
75.21
$0.21 (-5679)
74.32
32


Reviews:
[{"review_id": "AQcaMn802y", "reviewer": "Reviewer_JjFG", "summary": "This paper deals with the test-time adaptation (TTA) task and extends it to a challenging scenario, i.e., zero-shot noisy test-time adaptation (ZS-NTTA). The ZS-NTTA  adapts the source model to deal with the noisy target data during test-time and allow the target data to be noisy. This work first studies which factors can decline the performance in the ZS-NTTA task, then proposes a new method to overcome these problems.", "questions": "1. Please discuss the reason to evaluate the methods using rank in Fig. 2. \n2. It is better to analyze the theoretical foundation of injecting the noise to the data using the proposed method.\n3. It is better to study how the portion of the noisy labels influence the performance.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work studies a challenging task of zero-shot noisy test-time adaptation, which is practical and applicable to in real-world applications.\n2. The three observations are interesting and novel. They reveals the reasons why the NS-TTA is difficult and contribute to the community.\n3. The proposed method is reasonable and achieve good results in the experiment.", "weaknesses": "1. The proposed method studies the ranking distribution of different methods in Fig. 2. Can these ranking really tell the superiority of these methods? It is possible that one method is good at dealing with challenging datasets, and the other is good at easier datasets. Why not evaluate them using the absolute accuracy (instead of the ranks in this work)?\n2. How to inject the noise to the data is an important factor of the proposed method. Is there any theoretical analysis?\n3. It is better to study how the portion of the noisy labels influence the performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QbzAOidjVx", "reviewer": "Reviewer_GcDg", "summary": "This paper introduces a novel task setting termed zero-shot noisy Test-Time Adaptation (ZS-NTTA), focusing on adapting models to target data containing noisy samples during test-time in a zero-shot manner. To address this challenge, the authors propose the AdaND method, which trains a noise detector online using detected noisy samples. The results of the experiments are reported in terms of classification accuracy and Out-of-Distribution (OOD) detection metrics.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed task addresses issues commonly encountered in real-world applications of classification models, which are often overlooked in current research. For instance, existing OOD detection methods assume a clean stream when reporting accuracy, and clip models are applied in noisy TTA tasks for zero-shot usage.\n2. Although the method of training an additional noise detector is simple, it proves to be quite effective according to the experimental results.\n3. There is improved performance in both classification and OOD detection metrics.", "weaknesses": "1. I have concerns regarding the definition of the task name. The ultimate goal of the proposed ZS-NTTA task is to detect OOD samples and correctly classify In-Distribution (ID) samples. I understand the definitions of noisy TTA and OOD detection, where both aim to handle test sets containing both ID and OOD cases. In my view, TTA leans more towards a methodological strategy focusing on enhancing model performance and adaptability during testing, while OOD detection leans towards a task definition, primarily studying how to handle test datasets with noise/OOD samples. Thus, from this perspective, the task might be more appropriately named something like \"test-time OOD detection,\" as there are similar task definitions in academia, such as [1,2], which require discussion and comparison.\n\n[1] Test-Time Linear Out-of-Distribution Detection, CVPR2024\n\n[2] Atta: Anomaly-aware test-time adaptation for out-of-distribution detection in segmentation. NIPS2023\n\n2. Assuming there are no issues with the paper's title or task definition, and considering that TTA focuses more on classification accuracy while OOD detection focuses on the capability of detecting OOD/noise, there appears to be a mismatch between the methodology and the title of the paper. Specifically, the method emphasizes training an OOD detector with a fixed classifier, focusing on OOD detection capability rather than improving classification accuracy. I suspect the improvement in classification accuracy reported in the paper's tables is due to successful OOD detection rather than enhancements to the classifier, correct?\n3. TTA methods are generally sensitive to the ratio of clean to noise in samples, ranging from 1:1000 to 1000:1. Is the performance of this method stable across such variations?\n4. Line 376 is ambiguous and could lead readers to misunderstand that the clean and noisy streams are known. If I understand correctly, the paper suggests that the test stream is unknown, and that inserting Gaussian noise can handle various unknown test stream scenarios, including both clean and noisy streams.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9WfKiHc3fy", "reviewer": "Reviewer_SvHV", "summary": "The paper introduces a novel setting called Zero-Shot Noisy TTA and builds benchmarks. It provides a theoretical analysis of why existing TTA methods struggle in noisy scenarios. The paper proposes AdaND, which decouples the classifier and detector to identify noisy data. Experiments show that AdaND works well in ZS-NTTA and achieves computational efficiency.", "questions": "As the authors analyze in Section 3.3, some inaccuracies during the early stages of TTA can gradually lead the model to overfit. I wonder whether the pseudo-labels in the early stages are accurate and the inaccuracies in the Noise Detector will accumulate.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-organized, and the idea of the Zero-Shot Noisy TTA setting is novel. It offers a comprehensive analysis of existing methods that suffer from Zero-Shot Noisy TTA, and this analysis is easy to follow. The results demonstrate a notable improvement across many benchmarks, and the proposed method AdaND is computationally efficient.", "weaknesses": "Though Zero-shot Noisy TTA is a new task, the method of utilizing two stages and pseudo labels to train a noisy detector is not novel; the idea of using unlabel test data to train a more robust OOD classifier has been explored in other scenarios, such as [1] for zero-shot OOD detection.\n\nIt would be better to provide more experiments with more complex datasets, such as near ID/OOD datasets, for example, Imagenet and NINCO[2].\n\nSome recent test-time adaptation works are not compared in the experiments, such as [4], which adopts a training-free adapter to achieve efficiency. I am concerned whether the noise detector can be replaced by another training-free module to be more efficient for test time and get better performance, such as cache memory.\n\n[1] Du, Xuefeng, et al. \"How Does Unlabeled Data Provably Help Out-of-Distribution Detection?.\" ICLR 2024\n[2] Bitterwolf, Julian, Maximilian Mueller, and Matthias Hein. \"In or out? fixing imagenet out-of-distribution detection evaluation.\"\n[3] Karmanov, Adilbek, et al. \"Efficient Test-Time Adaptation of Vision-Language Models.\" In CVPR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)