Noisy Test-Time Adaptation in Vision-Language Models

Authors: Chentao Cao, Zhun Zhong, Zhanke Zhou, Tongliang Liu, Yang Liu, Kun Zhang, Bo Han
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: OOD Detection, Test-Time Adaptation

Abstract:
Test-time adaptation (TTA) aims to address distribution shifts between source and target data by relying solely on target data during testing. In open-world scenarios, models often encounter noisy samples, i.e., samples outside the in-distribution (ID) label space. Leveraging the zero-shot capability of pre-trained vision-language models (VLMs), this paper introduces Zero-Shot Noisy TTA (ZS-NTTA), focusing on adapting the model to target data with noisy samples during test-time in a zero-shot manner. In the preliminary study, we reveal that existing TTA methods suffer from a severe performance decline under ZS-NTTA, often lagging behind even the frozen model. We conduct comprehensive experiments to analyze this phenomenon, revealing that the negative impact of unfiltered noisy data outweighs the benefits of clean data during model updating. In addition, as these methods adopt the adapting classifier to implement ID classification and noise detection sub-tasks, the ability of the model in both sub-tasks is largely hampered. Based on this analysis, we propose a novel framework that decouples the classifier and detector, focusing on developing an individual detector while keeping the classifier (including the backbone) frozen. Technically, we introduce the Adaptive Noise Detector (AdaND), which utilizes the frozen model's outputs as pseudo-labels to train a noise detector for detecting noisy samples effectively. To address clean data streams, we further inject Gaussian noise during adaptation, preventing the detector from misclassifying clean samples as noisy. Beyond the ZS-NTTA, AdaND can also improve the zero-shot out-of-distribution (ZS-OOD) detection ability of VLMs. Extensive experiments show that our method outperforms in both ZS-NTTA and ZS-OOD detection. On ImageNet, AdaND achieves a notable improvement of $8.32\%$ in harmonic mean accuracy ($\text{Acc}_\text{H}$) for ZS-NTTA and $9.40\%$ in FPR95 for ZS-OOD detection, compared to state-of-the-art methods. Importantly, AdaND is computationally efficient and comparable to the model-frozen method. The code is publicly available at: https://github.com/tmlr-group/ZS-NTTA.

Decision: accept (poster)
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730187383919

thread_type:
official_review

<thread_object>
creator: reviewer_svhv

<content>
summary:
The paper introduces a novel setting called Zero-Shot Noisy TTA and builds benchmarks. It provides a theoretical analysis of why existing TTA methods struggle in noisy scenarios. The paper proposes AdaND, which decouples the classifier and detector to identify noisy data. Experiments show that AdaND works well in ZS-NTTA and achieves computational efficiency.

strengths:
The paper is well-organized, and the idea of the Zero-Shot Noisy TTA setting is novel. It offers a comprehensive analysis of existing methods that suffer from Zero-Shot Noisy TTA, and this analysis is easy to follow. The results demonstrate a notable improvement across many benchmarks, and the proposed method AdaND is computationally efficient.

weaknesses:
Though Zero-shot Noisy TTA is a new task, the method of utilizing two stages and pseudo labels to train a noisy detector is not novel; the idea of using unlabel test data to train a more robust OOD classifier has been explored in other scenarios, such as [1] for zero-shot OOD detection.

It would be better to provide more experiments with more complex datasets, such as near ID/OOD datasets, for example, Imagenet and NINCO[2].

Some recent test-time adaptation works are not compared in the experiments, such as [4], which adopts a training-free adapter to achieve efficiency. I am concerned whether the noise detector can be replaced by another training-free module to be more efficient for test time and get better performance, such as cache memory.

[1] Du, Xuefeng, et al. "How Does Unlabeled Data Provably Help Out-of-Distribution Detection?." ICLR 2024
[2] Bitterwolf, Julian, Maximilian Mueller, and Matthias Hein. "In or out? fixing imagenet out-of-distribution detection evaluation."
[3] Karmanov, Adilbek, et al. "Efficient Test-Time Adaptation of Vision-Language Models." In CVPR 2024.

questions:
As the authors analyze in Section 3.3, some inaccuracies during the early stages of TTA can gradually lead the model to overfit. I wonder whether the pseudo-labels in the early stages are accurate and the inaccuracies in the Noise Detector will accumulate.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730344514353

thread_type:
official_review

<thread_object>
creator: reviewer_gcdg

<content>
summary:
This paper introduces a novel task setting termed zero-shot noisy Test-Time Adaptation (ZS-NTTA), focusing on adapting models to target data containing noisy samples during test-time in a zero-shot manner. To address this challenge, the authors propose the AdaND method, which trains a noise detector online using detected noisy samples. The results of the experiments are reported in terms of classification accuracy and Out-of-Distribution (OOD) detection metrics.

strengths:
1. The proposed task addresses issues commonly encountered in real-world applications of classification models, which are often overlooked in current research. For instance, existing OOD detection methods assume a clean stream when reporting accuracy, and clip models are applied in noisy TTA tasks for zero-shot usage.
2. Although the method of training an additional noise detector is simple, it proves to be quite effective according to the experimental results.
3. There is improved performance in both classification and OOD detection metrics.

weaknesses:
1. I have concerns regarding the definition of the task name. The ultimate goal of the proposed ZS-NTTA task is to detect OOD samples and correctly classify In-Distribution (ID) samples. I understand the definitions of noisy TTA and OOD detection, where both aim to handle test sets containing both ID and OOD cases. In my view, TTA leans more towards a methodological strategy focusing on enhancing model performance and adaptability during testing, while OOD detection leans towards a task definition, primarily studying how to handle test datasets with noise/OOD samples. Thus, from this perspective, the task might be more appropriately named something like "test-time OOD detection," as there are similar task definitions in academia, such as [1,2], which require discussion and comparison.

[1] Test-Time Linear Out-of-Distribution Detection, CVPR2024

[2] Atta: Anomaly-aware test-time adaptation for out-of-distribution detection in segmentation. NIPS2023

2. Assuming there are no issues with the paper's title or task definition, and considering that TTA focuses more on classification accuracy while OOD detection focuses on the capability of detecting OOD/noise, there appears to be a mismatch between the methodology and the title of the paper. Specifically, the method emphasizes training an OOD detector with a fixed classifier, focusing on OOD detection capability rather than improving classification accuracy. I suspect the improvement in classification accuracy reported in the paper's tables is due to successful OOD detection rather than enhancements to the classifier, correct?
3. TTA methods are generally sensitive to the ratio of clean to noise in samples, ranging from 1:1000 to 1000:1. Is the performance of this method stable across such variations?
4. Line 376 is ambiguous and could lead readers to misunderstand that the clean and noisy streams are known. If I understand correctly, the paper suggests that the test stream is unknown, and that inserting Gaussian noise can handle various unknown test stream scenarios, including both clean and noisy streams.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730509986264

thread_type:
official_review

<thread_object>
creator: reviewer_jjfg

<content>
summary:
This paper deals with the test-time adaptation (TTA) task and extends it to a challenging scenario, i.e., zero-shot noisy test-time adaptation (ZS-NTTA). The ZS-NTTA  adapts the source model to deal with the noisy target data during test-time and allow the target data to be noisy. This work first studies which factors can decline the performance in the ZS-NTTA task, then proposes a new method to overcome these problems.

strengths:
1. This work studies a challenging task of zero-shot noisy test-time adaptation, which is practical and applicable to in real-world applications.
2. The three observations are interesting and novel. They reveals the reasons why the NS-TTA is difficult and contribute to the community.
3. The proposed method is reasonable and achieve good results in the experiment.

weaknesses:
1. The proposed method studies the ranking distribution of different methods in Fig. 2. Can these ranking really tell the superiority of these methods? It is possible that one method is good at dealing with challenging datasets, and the other is good at easier datasets. Why not evaluate them using the absolute accuracy (instead of the ranks in this work)?
2. How to inject the noise to the data is an important factor of the proposed method. Is there any theoretical analysis?
3. It is better to study how the portion of the noisy labels influence the performance.

questions:
1. Please discuss the reason to evaluate the methods using rank in Fig. 2. 
2. It is better to analyze the theoretical foundation of injecting the noise to the data using the proposed method.
3. It is better to study how the portion of the noisy labels influence the performance.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732158991293

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
A General Response by Authors

comment:
**We would like to thank all the reviewers for their thoughtful suggestions on our paper.** 

**We have received three reviews with ratings 6, 6, 5, and we are glad that all the reviewers have good impressions of our work**, including **(1)**  proposing a novel and practical setting (JjFG, GcDg, SvHV); **(2)** a simple, reasonable, and effective method (JjFG, GcDg);  **(3)** a comprehensive and easy to follow analysis with novel observations of existing TTA methods (JjFG, SvHV).

**In the rebuttal period, we have provided detailed responses to all the comments and questions point-by-point.** Following the reviewers' suggestions, we have revised the paper accordingly. Our main response can be summarized as:
 -  Clarify why we use ranking distribution in Fig. 2 and add absolute accuracy to evaluate TTA methods (Weakness1 for JjFG);
 - Clarify our ZS-NTTA setting, compare and discuss with test-time OOD detection (Weakness1 for GcDg);
 -  Clarify the connection between our method and our task (Weakness2 for GcDg);
 - Add experiments with more different noise ratios (Weakness3 for GcDg);
- Clarify the novelty of our method lies in decoupling the classifier and detector according to our analysis (Weakness1 for SvHV);
- Provide more experiments with more complex datasets (Weakness2 for SvHV);
- Conduct comparisons with TDA (Weakness3 for SvHV);
- Furthermore, we have updated our draft according to the reviewers' comments.

Lastly, we would appreciate all reviewers’ time again. Would you mind checking our response and confirming whether you have any further questions? **We are anticipating your post-rebuttal feedback!**
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces a novel research problem called zero-shot noisy test-time adaption, which aims to handle both ID and OOD test samples. The proposed method, AdaND, is simple yet effective, showing effective performance in classification and OOD detection metrics. Reviewers raised concerns about the definition of the task, technical details, experiments, and writing. The author's rebuttal successfully addressed the majority of concerns raised by the reviewers. At the end of the rebuttal, all reviewers agree to accept this paper, putting this paper above the acceptance bar.