Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Test-time adaptation updates a model to reduce generalization on shifted data.
Such adaptation methods need to choose a loss for adaptation and parameters to update, and this work's main contribution is to introduce visual prompts as a parameterization.
The proposed Data-efficient Prompt Tuning (DePT) method includes additional learnable tokens in a ViT during training, then updates these tokens and the classification token during testing, by entropy minimization through pseudo-labeling and other losses.
The adaptation parameters are limited to the classification head and the proposed layer-wise prompts, which strikes a balance between expressivity and efficiency, where the number of prompts can be varied from just one layer to all layers.
This parameterization is justified by the accuracy achieved (Table 1) and analysis experiments that try alternatives like updating layer normalization parameters (Appendix B.1).
The other losses include a self-supervised regularization scheme derived from DINO, in which projections of the prompts (local representations) and the class token (the global representation) are updated to maximize their similarity.
Experiments evaluate DePT against the prior state-of-the-art AdaContrast, reproduced with the same ViT backbone as the proposed method, and other test-time methods with ResNet backbones (which are therefore not comparable), along with unsupervised domain adaptation methods that require both source and target data.
DePT achieves higher accuracies by offline adaptation on the whole test set and online adaptation on the streaming test set, with a larger margin of improvement for online adaptation.
These results are shown for domain adaptation and generalization datasets, VisDA-C and DomainNet-126, but not corruption or other robustness benchmarks like ImageNet-C or ImageNet-R.
Ablation studies show that each component of the method helps.
Although pseudo-labeling makes the largest difference, and that is already a popular part of test-time adaptation, it is pseudo-labeling as a loss to update the proposed parameterization by prompts that is helping so much.


strength_and_weaknesses: *Strengths*:

- This work brings test-time adaptation up-to-date w.r.t. current model architectures and parameters: ViTs are current, strong models and prompting is a timely topic.
  The choice of parameterization is novel for this purpose, low-dimensional for efficiency, and specifically-chosen for ViTs.
- Online adaptation accuracy improves on the prior state-of-the-art AdaContrast by 4 points on VISDA-C and 2 points on DomainNet (which are both standard benchmarks).
  Offline adaptation accuracy is no worse than the state-of-the-art and perhaps slightly better.
- The choice of datasets is sound. VISDA-C is a standard benchmark, and DomainNet is a more recent benchmark that has also seen adoption. The evaluation protocol for these datasets follows prior work.
- There is a self-supervised loss that does not depend on the pseudo-labels, for bottom-up adaptation to the target inputs, which may complement top-down adaptation to the model predictions.
  This loss is closely-related to DINO, which is credited accordingly, but it is somewhat customized to the use of visual prompts in this work.

*Weaknesses*:

- While DePT efficient in its parameter dimension, that does not mean it is efficient in its update computation. For DePT, the prompts are distributed throughout the model, and so optimization has to compute many gradients across layers even if the parameter gradients are themselves much smaller.
  The amount of computation required during testing is not described, nor is it compared against baselines.
- There is only one comparable baseline with the same model architecture (AdaContrast). For thoroughness, it would be useful to adapt a prior method that reported results with ResNets to ViTs for comparison. Tent for instance simply minimizes the entropy of predictions for a given choice of parameters, and could be ported to ViTs by updating the layer norm parameters (for one example).
- There are no results for other common choices of robustness or domain generalization benchmark, such as ImageNet-C (or its other variations ImageNet-V2, ImageNet-R, etc.), or PACS/VLCS/etc. These are included in many prior works such as TENT, SHOT, and TTT.
- There is missing related work on learning and tuning visual prompts, although neither addresses the application of visual prompts to test-time adaptation against shifts, as done in this work.
  - [Interactive Image Segmentation via Backpropagating Refinement Scheme](https://vcg.seas.harvard.edu/publications/interactive-image-segmentation-via-backpropagating-refinement-scheme/paper). Jang & Kim, CVPR'19. This paper prefigures visual prompts by perturbing the input according to a model's own predictions as a kind of self-training to improve accuracy. Citing it would help better trace the history of ideas.
  - [Exploring Visual Prompts for Adapting Large-Scale Models](https://arxiv.org/abs/2203.17274). Bahng et al. arXiv'22. This is highly related to the cited VPT by Jia et al. 2022 and should be considered for reference alongside it.
- The proposed method is not fully test-time because the prompts must be jointly trained with the source model parameters. Its setting should be better identified in the text, and the experiments should compare to the latest source-free methods accordingly, like SHOT++ (Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer. Liang et al. PAMI'21.)


clarity,_quality,_novelty_and_reproducibility: *Clarity*:

- This work can be understood, but it could be more accessible. The exposition would be improved by making introductory statements more definite and citing established vocabulary when it is first used. There are grammatical errors that take time to read through, so proofreading would also make this work more readable.
- The introduction does not fully explain the setting. It should define the test-time adaptation setting and contrast it with source-free adaptation, as the two are sometimes confused, and the established unsupervised domain adaptation setting, which is often synonymous with "domain adaptation" in the literature.
- The summarization of the memory bank in Figure 2(B) does not adequately explain the aggregated prompt or how the contents of the memory bank are initialized and updated. The caption should explain the multiple losses, if only briefly, and indicate how the memory bank alters the pseudo-labels.
- Examples are relegated to the appendix, such as appendix B.1, which would better serve the reader as part of the main paper. B.1 in particular has key motivating results for visual prompts as parameters.
- Overall the writing, figures, and tables are adequate for communicating the method and results, with only a few exceptions.

*Quality*:

- The results improve on standard benchmarks, by as much as +4 points when comparing to the prir state-of-the-art and controlling for backbone, which is a larger margin than is sometimes reported in papers on test-time adaptation (where improvements may be +1 or +2 points).
- The method is sensible, as visual prompting has been shown to work elsewhere such as for transfer learning, and the experiments justify this choice of parameterization for the purpose of test-time adaptation.
- The related work could better credit other visual prompting work that has been done (see weaknesses), but at least the application contributed here for test-time adaptation is executed well and delivers an improvement.

*Novelty*:

- Test-time adaptation to shift is a novel application of visual prompts in the form of additional learnable tokens. The experiments on choices of parameterization for test-time adaptation are empirically novel and informative.
- The pseudo-labeling update (Section 3.2.1) is not novel, as moving statistics in the form of exponential moving averages and student-teacher updates are common for self-supervision (MoCo, ODIN, ...) and adaptation (SHOT, DINE, ...).
- The self-supervision update (Section 3.2.2) is essentially DINO, as cited, but it has a few of its own implementation details including a diversity regularizer across the leraned prompts.
- Adapting to smaller amounts of data (Figure 3) is not novel, as claimed in the abstract, because online test-time adaptation methods already only adapt to the data given for testing.
  The significance of its insensitivity to the amount of data for adaptation is also more theoretical than practical, as knowing when to adapt or not is itself a problem, which is why prior methods like TTT or Tent keep adapting.

*Reproducibility*:

The explanation and appendices have a fair amount of detail, but given the number of terms in the loss and variety of update schemes (teacher-student, online memory, EMA, etc.) this work may not be reproducible from the paper alone.
There is no statement about releasing the code.


summary_of_the_review: In short, the method does work as shown by its evaluation on the standard datasets of VisDA-C and DomainNet.
However, the evaluation could be broader in its datasets and more thorough in its comparable baselines by reproducing more than one method with the ViT backbone.
This is an expectation for work in this area, as many past test-time adaptation papers have evaluated on several datasets of shifts as a way to shown some measure of generality.
All in all, there is value in demonstrating that test-time adaptation applies to ViT models too, and in exploring the right choice of parameterization for this purpose.
While this work could be improved, pushing test-time adaptation to use the most accurate architectures at present may be sufficiently informative to the community.

*For Rebuttal*

1. Please measure the time efficiency of DePT updates, and relate the computation required to update the prompts to the computation required to update normalization layers or other choices of parameters, as used by the compared methods like TENT and SHOT.
2. Please discuss DePT as a source-free (SHOT) vs. test-time (TENT) vs. intermediate (TTT) method. The choice of setting should guide the choice of comparisons.
3. Please report results for ImageNet-C, if possible, because it is a common benchmark for robustness and adaptation to corruption. (It is alright if this is not possible due to computational limitations, but doing it would be a plus.)
4. Please motivate the low data regime reported in the results. Is there a practical deployment that this setting addresses?

*Other Feedback*

- Title
  - Is the method a test-time adaptation method? Test-time methods update during testing, as DePT does, but in some usages "test-time" also indicates that the method can update online and that it does not require changes to training. DePT seems to be more like SHOT or TTT in that it alters training.
- Abstract
  - Name the datasets, benchmarks, and settings evaluated. A precise summary of experiments is more eye-catching to the potential reader.
- Related Work
  - The formatting with subsections takes up a lot of space. Consider revising it to remove the subsection headings and use more bolded heading instead. Then the paper would have more room for further explanation or experiments.
- Analysis
  - Table 3 is not an impressive comparison, because DePT has been augmented to include domain-specific and domain-shared parameters while SHOT has not been. Other methods like SHOT could also have domain-specific layers, as done by residual adapters (Rebuffi et al.) for example.
  - Table 4 should include the accuracy without DePT in the caption to help ground the relative improvement of each number of prompts.
- Proofreading
  - "Second, given only unlabeled target domain data, use what kind of learning objective for optimization." Should this be a question? Please revise.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a test-time adaptation method based visual prompt tuning. After training the deep model along with a set of prompts to convergence, the proposed DePT fixes the backbone parameters and only fine tunes the prompts on the target domain. Based on this learning scheme, DePT integrates multiple self-training techniques (with adequate modification) into the training objective, including a pseudo labeling under the student-teacher framework, modified DINO, consistency on both the class token output (through cross-entropy loss) and the prompt token output (through mean squared error). Experimental results validate the effectiveness and investigate the major components through ablation. 
strength_and_weaknesses: **Strength**

- Using an efficient fine-tuning technique, i.e., prompt tuning for test-time adaptation is technically sound. 

- The achieved accuracy is competitive.


**Weakness**

- Introduction for some important components lacks intuition and is not easy to understand, e.g., the memory-based online pseudo labeling in the abstract and the introduction. It not easy to understand its name and how it performs until the detailed method. 

- While the training objectives are modified from recent self-supervised learning techniques, the authors have not provided necessary comparison or revisit. For example, the pseudo labeling part can be viewed as a modification from MoCo, but the authors introduce the momentum teacher as if it was invented for the first time. Moreover, the MSE losses in Eq. 4~5 (and their input features) have strong relation to BYOL.

- Based on the above concern, it is important to explain why the proposed method makes those modifications (instead of directly applying the original MOCO or BYOL). 


clarity,_quality,_novelty_and_reproducibility: - The paper is generally well written and easy to understand (except that some concepts need better explanation at their early appearance, e.g., the memory-based online pseudo labeling). 

- Prompt tuning is a popular technique. However, using prompt tuning for test-time domain adaptation has some novelty. 

summary_of_the_review: I am generally satisfied with the contribution of this paper. I would like to raise my rating if the authors could address my concerns above, especially the relation between their training objectives and the original self-supervision methods.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper present a method for test time adaptation using learnable visual prompts integrated with visual transformers.  The paper uses various regularization methods to avoid error accomulation; knowledge distilation from ensemble or sourch checkpoints, self-supervised learning and learnable visual prompts. 
The results are very promising. 
strength_and_weaknesses: strengths:
- The paper is easy to read and the flow of the paper is generally good. 
- Figure 1 is highlights an important and intersting experiment with the number of available target examples in the semi-supervised learning setting. 
- Results are very promising

weaknesses:
- I think there needs to be discussion and ablation study on the importance of visual prompts. They describe how they are trained but whats missing is a discussion on why there are needed. 
- The paper misses some important citations. For example "hypothesis disparity regularized mutual information maximization" AAAi 2020 uses ensembles for regularization of TTA which is closely related to the proposed method.  
- There needs to be more dicussion on what the self-supervised learning is set to achieve differently from that of the pseudo label training. Also what was the process of desing choice for selecting DINO over other SSL methods? 
-Error bars (repeated experiments with different parameter initialization seeds) are missing from expeirments which makes it hard to evaluate reproducibility and robustness. 

clarity,_quality,_novelty_and_reproducibility: - the paepr is generally easy to understand. 
- The paper does not propose any novel theory  but rather gathers ideas from other places and applies them to TTA which is okay. 

in terms of reproducibility and robustness, its not clear how the hyper-parameters for equation 6 (the loss function) where chosen?. Do we assume we have access to a targe validation set (sebset of the target data with labels)

another question about clarity is the experimental setup for figure 1. I assume in figure 1 , the size of the test data is different for points in the horizantal axis and thus they arent exactly comparable. 

Error bars (repeated experiments with different parameter initialization seeds) are missing from expeirments which makes it hard to evaluate reproducibility and robustness. 

Whats the difference between strong transform and weak transform? Has there been any ablation study or discussion on why strong transform is used in some places and weak transform in other places? 
summary_of_the_review: The paper is interesting however, it misses some references (see above) and I cant seem to find an original point for novelty but the experimental results show significant improvement over the current state of the art. I hope the authors provide clarifications in the rebuttal phase.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper considers the problem of test-time domain adaption. Specifically, the authors propose a data-efficient prompt tuning (DePT) approach, which only requires finetuning injected visual prompts and classifiers during training. In addition, a memory-based pseudo-labeling method is also introduced to improve the quality of pseudo-labels. Experiments on several datasets show the benefit of the proposed DePT.


strength_and_weaknesses: 
Strength

+ This paper is well-written and easy to follow. The motivation is clear and reasonable.

+ Clear figures are provided to help us better understand the details of the proposed method.

+ The proposed diversity term is novel and interesting.

+ Extensive experiments are provided to evaluate the effectiveness of the proposed method.

Weakness

+ The novelty of the proposed method is somewhat limited. The proposed method includes two parts, memory-based pseudo-labeling and visual prompt learning [A]. However, both parts are already studied in the domain adaptation community, especially memory-based pseudo-labeling.

+ Although the authors argue that the proposed method achieves significant improvement, the improvement is mainly obtained by pseudo-labeling (Table 5). The CLS regression can also improve the performance, but it is largely bought from DINO, i.e., distillation with teacher-student strategy. From Table 5, we can find that the improvements of the proposed memory-based PL and prompt tuning are somewhat limited.

+ To verify the effectiveness of prompt tuning, it is better to show the ablation study of each component in more datasets.

+ Although the proposed method requires few parameters to be tuned compared to other methods, I think the computation cost and training time are not reduced too much. This is because the prompts are injected into different layers, and the gradients should be calculated on all layers. In other words, tuning fewer parameters does not mean saving computation costs and training time. I thus think it is required to provide the computation cost (memory and training time) for different methods.

+ How to ensure that the injected prompts could contain domain-specific information? In addition, it would help us better understand the learned prompts by visualization as shown in Figure 3.

+ In the recent community, the test-time domain adaptation commonly refers to the online setting. While the experiments mostly conducted in this paper should be called source-free domain adaptation or test-time training adaptation. Please clarify them in the next version.

[A] Domain Adaptation via Prompt Learning.

clarity,_quality,_novelty_and_reproducibility: 
Clarity and Quality: This paper is well-written and easy to follow. The description of the proposed method, the figures and the tables are clear. 

Novelty: The novelty of the proposed memory-based PL and prompt tuning are somewhat limited. However, the diversity term is interesting.

Reproducibility: No source code is provided.


summary_of_the_review: This paper is well-written and easy to follow.  However, the novelty and improvement of the proposed method are limited. The computation cost is not evaluated to show the real benefit of the proposed method compared to other methods. Also, this paper lacks a convincing explanation for the prompt tuning. Considering these factors, I tend to give borderline reject.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a test-time unsupervised domain adaptation (training unlabeled target data without access to the source data) method called DePT, based on visual prompt-tuning based ViT models. Experimental results show that the proposed DePT outperforms other ViT based methods. 

Strength:
1. Presentation is clear
2. Applying visual prompt-tuning ViT in test-time UDA is novel

Weakness:
1. Though given Strength 2, the authors didn't provide any insights into why using such ViT models is especially useful for test-time UDA. For example, visual prompt is essentially a set of visual priors learned from ViT, why can such priors encode informative representations of source data?

2. The experimental comparisons are not carefully designed. For example, comparison with the traditional non-test time  UDA is non-informative; Ablations should be more emphasized on the visual prompt based ViT.
justification_for_why_not_higher_score: This paper does not bring any new insight into the community. It simply deploys a method in a task. The weakness clearly outweighs the strength.
Decision: Reject