Intransigent Teachers Guide Better Test-Time Adaptation Students

Authors: Damian Sójka, Marc Masana, Bartłomiej Twardowski, Sebastian Cygert
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: test-time adaptation

Abstract:
Test-Time Adaptation (TTA) has recently emerged as a promising strategy that allows the adaptation of pre-trained models to changing data distributions at deployment time, without access to any labels. To address the error accumulation problem, various approaches have used the teacher-student framework. In this work, we challenge the common strategy of setting the teacher weights to be an exponential moving average of the student by showing that error accumulation still occurs, but only on longer sequences compared to those commonly utilized. We analyze the stability-plasticity trade-off within the teacher-student framework and propose to use an intransigent teacher instead. We show that not changing any of the weights of the teacher model within existing TTA methods allows them to significantly improve their performance on multiple datasets with longer scenarios and smaller batch sizes. Finally, we show that the proposed changes are applicable to different architectures and are more robust to changes in hyper-parameters.

Decision: reject
Reviewer scores: 5, 8, 3, 5

Reviews and discussion:
<thread>
creation date:
1729490438087

thread_type:
official_review

<thread_object>
creator: reviewer_3dwc

<content>
summary:
The paper challenges the existing teacher-student framework in test-time adaptation (TTA), where lifelong adaptations showed inevitable performance degradation. The paper proposes an intransigent teacher, which does not update the parameters but only uses test batch statistics. The intransigent teacher showed high stability in lifelong adaptation while improving performance compared to the original teacher-student-based methods.

strengths:
- Writing is clear, comprehensive, and easy to understand.

- Proposed a simple yet effective solution for the practical scenario of lifelong adaptation.

- Extensive large-scale evaluations on various datasets/scenarios and state-of-the-art baselines.

weaknesses:
- The problem (model failures in lifelong adaptation) has already been discussed in RDumb, so the problem setting itself is not novel.

- The method only applies to existing teacher-student methods, thus limiting its applicability. At the same time, the intransigent teacher does not consistently outperform the baselines (e.g., RDumb in BS=64) or prevent failures (e.g., results in BS=10).

questions:
- Please discuss the advantages/disadvantages of the proposed intransigent teacher compared to the important lifelong baseline, RDumb.

- Can we dynamically adjust the plasticity ($\beta$) to climb up to 1 (e.g., using the TTA accuracy estimation metrics [a, b] or using a fixed period)?

- Would this phenomenon also occur in non-corrupted lifelong test streams?

- Reporting single-pass results (akin to the original TTA setup) would help understand the performance compared to existing TTAs.

- (Minor) Typo: Page 8, Line 423: COTTA -> CoTTA

---

[a] Lee, Taeckyung, et al. "AETTA: Label-Free Accuracy Estimation for Test-Time Adaptation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[b] Kim, Eungyeup, et al. "Reliable Test-Time Adaptation via Agreement-on-the-Line." arXiv preprint arXiv:2310.04941. 2024.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730627253076

thread_type:
official_review

<thread_object>
creator: reviewer_3kuq

<content>
summary:
An interesting phenomenon found in the paper is that the intransigent teacher model is able to guide a more stable student model in long sequences of CTTA tasks. From the conclusions in the paper, it is clear that this approach can be applied to all methods of the mean-teacher architecture.

strengths:
The phenomenon observed in the paper does to some extent replace the current method of the mean-teacher architecture and can achieve better results in long sequences of CTTA tasks.

weaknesses:
The text lacks critical experimental and theoretical proofs and does not present targeted methods and analyses.

Limitations：
1. The paper does not provide a detailed analysis, but is only based on experimentally observed phenomena, and it is not possible to determine the specific reasons for the decline in generalizability of the teacher model, nor does it give a specific analysis of the decline in generalizability performance of the teacher model.
2. Why the intransigent teacher model outperforms the EMA updated teacher model in the long sequence CTTA task, relying only on experimental comparisons is not convincing.
3. Is the Intransigent teacher model just setting β to 1? How is this different from freezing the model? Is it understood to always use the source model as the teacher model? If so, it is no longer considered to be a mean-teacher framework.
4. Should the teacher model be locked in any scenario? It is suggested that the authors consider a scenario where the weights of the teacher model are dynamically adjusted, which might achieve better results.
5. Based on the phenomena you observed, the paper doesn't seem to suggest any targeted approach? Does this imply that you are just using the source model as a teacher model? I don't see any relevant methods in the source code either.
6. Although comparisons were made on three methods in the paper, the paper should have added more comparison experiments with the Mean-Teacher architecture method. Also, the authors need to provide segmentation experiments to further demonstrate the effectiveness of their proposed approach.

questions:
See the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730646797661

thread_type:
official_review

<thread_object>
creator: reviewer_iald

<content>
summary:
This paper addresses the challenges of Test-Time Adaptation (TTA) and proposes to use a static (intransigent) teacher model, which does not update its weights during adaptation. The authors demonstrate that this modification enhances performance across multiple datasets characterized by longer sequences and smaller batch sizes. Additionally, they provide evidence that their proposed method is adaptable across various model architectures and exhibits robustness against changes in hyper-parameters.

strengths:
1.  The "intransigent teacher" concept is a fresh perspective that challenges existing methodologies in TTA, potentially leading to improved performance in real-world applications, such as LLM applications.
2. The authors support their claims with experimental results across multiple datasets, demonstrating the effectiveness of their approach in diverse scenarios. The proposed method shows robustness to hyper-parameter variations. 
3. The proposed approach is simple and can be generalized across different architectures.

weaknesses:
1. While the empirical results are compelling, the theoretical justification for why the intransigent teacher improves performance could be elaborated further to enhance the understanding of the underlying mechanisms.
2. The implications of using an unchanging teacher model over extended periods or across highly variable data distributions could be discussed more thoroughly, as this might lead to stagnation in learning.

questions:
See the weaknesses for details.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730686531934

thread_type:
official_review

<thread_object>
creator: reviewer_akal

<content>
summary:
The paper proposes an intransigent teacher (IT) based approach for continual test-time adaptation (TTA), where the teacher model is kept frozen, and only the student model updates. The aim is to alleviate the problem of error accumulation that is persistent in longer horizons of target domains.
Experimental results on longer horizons of corruption sequences demonstrate that IT helps improve performance in compared settings on multiple benchmarks.

strengths:
* Experiment with different approaches that use losses, such as consistency and contrastive losses.
* Improving performance on longer horizons on multiple benchmarks

weaknesses:
* Limited novelty. EMA-based continual TTA approaches already have a hyperparameter that decides how much weightage to be given to the student model weights and updates the teacher weights as a linear combination. If the weightage to student model weights is extremely low, it is effectively an "intransigent teacher."
* CoTTA [1] and PETAL [2] have already proposed a resetting mechanism that preserves source knowledge by resetting some weights back to the source pre-trained model.
* Repeated loops of the same data showing poor performance can also mean that the model is overfitting to each target domain and drifting away from source knowledge, which is suitable for all the target domains. Approaches such as CoTTA [1] and PETAL [2] have a resetting mechanism that consists of a threshold hyperparameter while resetting. Tuning this hyperparameter is essential for longer horizons using the validation corruption data. Otherwise, the comparison with baselines is not fair.
* The proposed approach is limited to EMA student-teacher models.

questions:
* If we refer to CoTTA paper [1] Equation 2 and its supplementary [2], \alpha (\beta in the submitted paper) can be put to 1, and it will effectively lead to an "intransigent teacher." Is this understanding correct? If so, what is the novelty of this paper, and why is it not just a trivial extension in terms of methodology?
* Is the paper simply not setting the \beta value to 1 and experiments around it?
* Is repeating the same corruption sequence multiple times realistic? If the paper claims intransigent teacher helps, there should be new benchmarks with longer horizons of corruption sequences, rather than repeating the corruption sequence.
* Tuning this hyperparameter is essential for longer horizons using the validation corruption data. Otherwise, the comparison with baselines is not fair. Have the authors tuned the hyperparameters for the baseline approaches? Also, was any validation corruption data used?

**References**
1. Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022
2. https://openaccess.thecvf.com/content/CVPR2022/supplemental/Wang_Continual_Test-Time_Domain_CVPR_2022_supplemental.pdf
3. Dhanajit Brahma, and Piyush Rai. A probabilistic framework for lifelong test-time adaptation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732579955123

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
First and foremost, we would like to thank all the reviewers for the insightful feedback on our work. We are particularly encouraged that reviewers *iaLD* and *3dWc* recognized the simplicity and effectiveness of our solution to the described problem. Additionally, reviewers *iaLD* and *3kuq* highlighted the broad applicability of our technique across various architectures and test-time adaptation methods based on the mean-teacher framework.

We have made the following changes to the revised version:
- Experiments with the new method (PETAL) with and without the Intransigent Teacher (Tables 3, A.8). The average improvement achieved by applying the intransigent teacher (IT) was 9.1% and 4.1% for batch sizes of 10 and 64, respectively.
- Results with parameter tuning (i.e., for baseline methods, Tables A.3, A.4, A.6),
- Results with increased plasticity by allowing the teacher to change for an initial fixed number of steps (Table A.5),
- Comparison with RDumb method (discussion and Fig. A.2).
- Results on source domain data streams (Table A.7).

Changes in text are indicated by the violet font color in the revised pdf version.

The results presented above highlight the robustness of using a fixed teacher, particularly in relation to variations in hyperparameters and corruption types. 

We welcome further discussions on any aspects of the paper that may require additional clarification. In particular, there are four recurring concerns raised by the reviewers, which we would like to address collectively:

**A. Limited Novelty. Method too simple, is it just setting β to 1.0?**

To make it absolutely clear, it is in fact setting the β value in the linear combination of weights to 1. This means that the weights of the teacher in the mean-teacher framework become fixed. Output predictions for the evaluation are taken from the student model, regardless of the method to which the IT is applied. One relevant detail is that we also update the statistics in batch normalization layers of the teacher model in the way the respective original base method did if the corresponding model has such layers.

We identify a significant limitation in current TTA methods, specifically their performance degradation on unusually long test sequences. This observation is valuable for the community as it highlights a crucial flaw in common evaluation protocols and SOTA methods utilizing the teacher-student framework.
In response to this issue, we offer a straightforward solution: the use of the IT. While simple, this approach effectively mitigates the described problem. Our primary goal is NOT to introduce a novel method that outperforms all existing SOTA approaches. Rather, we aim to:

1. Draw attention to the performance issue in current TTA methods,
2. Investigate commonly used teacher-student framework and its shortcomings,
3. Address the issues with a simple method, even though it is sometimes outperformed by other SOTA methods.

The novelty of our work lies in identifying the performance of degradation issue on extended test sequences and proposing a simple yet effective solution to mitigate this specific problem within the widely utilized teacher-student framework.

**B. Lack of theoretical justification.**

While our current study lacks a comprehensive theoretical framework, we present comprehensive experimental evidence documenting this previously unreported phenomenon. Our goal is to bring attention to these findings and establish a good starting point for future research as we have not yet uncovered the underlying theoretical justifications. We welcome the reviewers' insights and suggestions, which could guide the investigations further.

**C. Adaptive value of β.**

Dynamic adjustment of the β parameter, guided by an appropriate heuristic, has the potential to outperform any fixed value. In the rebuttal, we investigate this approach by allowing the teacher to adjust for a fixed number of steps initially (for a single loop), inspired by a recent influential Continual Learning paper [R1]. The results, presented in Table A.5, show that this approach yields slightly better outcomes compared to keeping the teacher fixed from the start. While this is not universally true across all settings, the findings highlight the promise of this method. Developing a more robust approach is left for future work, as determining the optimal timing for freezing the teacher requires careful tuning and hyper-parameter selection remains a challenging aspect of test-time adaptation.

[R1] Panos, Aristeidis, et al. "First session adaptation: A strong replay-free baseline for class-incremental learning." ICCV, 2023.

**D. The IT is only applicable to teacher-student framework-based methods.**

The EMA student-teacher models are a common approach in TTA and we show how it can have unwanted behavior in a reasonable extension of the current settings. Moreover, a universal SOTA method is not a contribution of this paper (See section A).
</content>
</thread_object>

</thread>
<thread>
creation date:
1733246844911

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
As the rebuttal period is nearing its end, we would like to summarize the current state of the rebuttal. Overall, the reviewers have acknowledged several strengths of the paper:

- **Extensive large-scale evaluations**: Conducted across diverse datasets, architectures and scenarios (iaLD, 3dWc) and loss functions (aKAL).
- **Problem importance**: Addressing the challenge of adaptation over long sequences (highlighted by reviewers aKAL and 3dWc) in practical scenarios (iaLD).
- **Broad applicability**: reviewers iaLD and 3kuq emphasized the versatility of our technique, which is applicable across various architectures and test-time adaptation methods based on the mean-teacher framework.
- **Simplicity and effectiveness**: as acknowledged by reviewers 3dWc and iaLD, with the latter also praising the method's robustness to changes in hyperparameters.
- **Presentation quality**: The paper received high scores for presentation, with the exception of reviewer 3kuq, who did not respond to our query regarding the reasons for their lower score.
    
The main weaknesses identified by the reviewers include the lack of theoretical justification (3kuq, iaLD), limited novelty (3dWc, aKAL), insufficient adaptive performance (3dWc, 3kuq), and concerns regarding hyperparameter selection (aKAL). During the rebuttal, we aimed to address the majority of the weaknesses raised by the reviewers through the following actions:

- **Successful application of IT**: We demonstrated its effectiveness on novel architectures (PETAL, as shown in Tables 3 and A.8).
- **Extensive hyperparameter search**: We explored additional tuning of hyperparameters (including CoTTA reset mechanism) (Tables A.3, A.4, and A.6).
- **Increased plasticity results**: We presented results where the teacher model was allowed to adapt for an initial fixed number of steps (Table A.5).
- **Novelty clarification with regard to RDumb paper**: We addressed this through detailed discussions with reviewer 3dWc and additional analysis in appendix (Section A.12).
- **Source domain data streams**: We provided results on adaptation to data streams without distribution shifts (Table A.7).

Finally, through further discussions with reviewers, we clarified:
- The necessity of using a fixed teacher model (instead of using student network only), addressing concerns from reviewer 3kuq.
- The use of batch statistics adaptation, as requested by reviewer 3dWc.
    
We would like to emphasize once again that the primary goal of the paper is the observation and analysis, rather than the IT method itself, which serves as a simple baseline for future work to "beat as a minimum." While we acknowledge the reviewers' point that the paper does not provide a theoretical explanation and ocassional underperformance on specific benchmarks, so do as many excellent machine learning papers that are empirically focused. We hope what we presented will spark discussion and inspire further investigation, maybe with more theorethical grounding as well. Most importantly, we believe the insights and performance analysis presented here will be valuable to the community and encourage further research on the limits of TTA methods.

Finally, we sincerely thank the reviewers for their initial feedback, which has significantly enhanced the quality of our work. We regret, however, that we have not received further responses from reviewers aKAL, 3kuq, and 3dWc following our latest revisions. This limits the opportunity for a more comprehensive discussion. However, we remain confident that the improvements implemented effectively address the concerns raised, and lead to higher scores from the reviewers whose feedback we have fully or partially addressed.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper presents a method for test-time adaptation (TTA) based on the student-teacher framework proposed in some other recent works on TTA. The paper argues that, unlike these recent works, not changing the teacher weights performs better. 

This paper received mixed score. The authors' response was discussed; however several of the reviewers' concerns remained. In particular, the insights reported by the paper are not very surprising (well-known in prior work) and the proposed solution doesn't seem to that competitive as compared to existing SOTA methods, raining concerns about the practical usefulness of the method. Therefore, the paper is slim in terms of providing new insights as well as providing a new method with improved performance. 

In view of these concerns, the paper doesn't appear to be strong enough to be accepted.