Paper: FEATHER: Lifelong Test-Time Adaptation with Lightweight Adapters (ICLR-2024)

Reviewer comment (Evaluation):
FEATHER lacks the strong objective. The proposed approach seems not to improve performance nor efficiency.

Author response:
**Performance comparison of FEATHER vs. CoTTA:** Table 3 clearly demonstrates that, for ImageNet-to-ImageNetC, FEATHER, with only 10.92% trainable adapter parameters, outperforms CoTTA (which adapts all (100%) of the network parameters) in terms of error rate. Further, Table 4 shows that FEATHER’s performance is **comparable** to CoTTA and other approaches, even **with only 6.71%** trainable adapters, while **outperforming** all these methods **with as few as 50.32%** trainable parameters (Table 5). FEATHER can be used with any desired number of trainable parameters depending on the computational budget. In Table 7 in Appendix of the old version (Now Table 4 in the main paper), FEATHER with CoTTA-based TTA objective outperforms EcoTTA for both CIFAR10-to-CIFAR10C and ImageNet-to-ImageNetC datasets. For ImageNet-to-ImageNetC datasets, FEATHER outperforms all the other approaches, including EcoTTA and CoTTA.

Reviewer comment (Evaluation):
From the performance perspective, FEATHER does not provide performance improvement over other baselines (e.g. CoTTA).

Author response:
**Performance comparison of FEATHER vs. CoTTA:** Table 3 clearly demonstrates that, for ImageNet-to-ImageNetC, FEATHER, with only 10.92% trainable adapter parameters, outperforms CoTTA (which adapts all (100%) of the network parameters) in terms of error rate. Further, Table 4 shows that FEATHER’s performance is **comparable** to CoTTA and other approaches, even **with only 6.71%** trainable adapters, while **outperforming** all these methods **with as few as 50.32%** trainable parameters (Table 5). FEATHER can be used with any desired number of trainable parameters depending on the computational budget. In Table 7 in Appendix of the old version (Now Table 4 in the main paper), FEATHER with CoTTA-based TTA objective outperforms EcoTTA for both CIFAR10-to-CIFAR10C and ImageNet-to-ImageNetC datasets. For ImageNet-to-ImageNetC datasets, FEATHER outperforms all the other approaches, including EcoTTA and CoTTA.

Reviewer comment (Evaluation):
From the efficiency perspective, while FEATHER updates a smaller percentage of network parameters when compared to CoTTA, FEATHER is less efficient than CoTTA. In essence, FEATHER adds extra parameters to the network making the forward pass more expensive. Further, the gradient calculation for conducting an update step of FEATHER and CoTTA are similar since FEATHER adds its adapters after every layer. Hence, the gradient will require a back propagation through the entire network.

Author response:
**Efficiency comparison of FEATHER vs CoTTA:** We believe the reviewer has missed a very important point here. The gradient calculation for conducting an update step of FEATHER and CoTTA is not similar. In FEATHER, the adapters are only present for some layers depending on the budget for the parameters. Thus, FEATHER does not introduce adapters after every layer. We have added the following line in section 3.1 to clarify this point: “In practice, we only insert the adapters in a few locations, depending on the computational and memory budget.”. Further, the **backpropagation only requires the computation of gradients of the adapter parameters and not the entire network.** (also see Appendix F for the architecture-specific adapter locations in detail.) Thus, FEATHER drastically improves the parameter efficiency over CoTTA. This is also reflected in the memory saving and wall-clock timing, which we have reported in the rebuttal (as requested by Reviewer CBjP) and in the updated version of the paper.

Reviewer comment (Experiments):
Since FEATHER is an orthogonal approach to TTA methods, why is it assumed that it does not need accessing source data? For example, if FEATHER is combined with EATA, source data is necessary for the anti-forgetting regularizer. Having said that, it is necessary to compare FEATHER to TTA methods that leverage source data.

Author response:
**Access to source data:** The initialization mechanism that we propose for FEATHER adapters circumvents the need for warm-up training of the adapters as in EcoTTA. Our approach does not inherently need source data for warm-up like EcoTTA. Our contribution is on the architectural aspect; however, if one wishes to use a learning objective like EATA, which depends on the source data for computing Fisher information, along with FEATHER, the source data would be utilized. However, this is a requirement of EATA and not that of FEATHER. We report the results of EATA + FEATHER in the updated version of the paper. (Table 6 and Table 7)

Reviewer comment (Evaluation):
Efficiency measures: The efficiency comparison in this work is based on the percentage of parameters being updated compared to the total number of parameters. It is unclear if this is the right way to compare different TTA methods. First, FEATHER adds extra parameters to the network, and thus by construction, its forward pass is slower than the baseline (e.g. CoTTA). A comparison in terms of runtime and memory usage is more necessary. It is worth mentioning that methods like Tent and EATA only update the normalization layers making them even more efficient than FEATHER.

Author response:
**Efficiency (memory/run-time) comparisons:** We have added the comparison in terms of runtime and memory usage in our common response. We provide detailed reasons for comparing the methods based on the number of trainable parameters.  We believe there is some misunderstanding regarding the results reported for TENT (the entire network). We would like to highlight that the TENT (100% trainable) is an additional setting run by us for a transparent comparison. The original TENT method using only BN params performs poorly in the continual setting due to error accumulation and forgetting over time.

Reviewer comment (Writing):
The mathematical notation and problem description is not clear. In section 2, \( f_\theta \) outputs the prediction \( \hat{y} \) at the beginning, and later in the same paragraph it is assumed to output a probability vector.

Author response:
**Writing:** Thanks for pointing out the places where you found the writing to be unclear. In particular, we would like to clarify that:
* By output, we meant the softmax probability vector (the predicted label corresponds to the index of the maximum value in the vector). We have clarified this.

Reviewer comment (Writing):
The notation in the last paragraph in Sections 3.1 and 3.2 is not clear. What is the element-wise addition? How is that different from a regular addition? What are the input and output shapes of the adapting layers?

Author response:
**Writing:** Thanks for pointing out the places where you found the writing to be unclear. In particular, we would like to clarify that:
* It should be an element-wise addition throughout; the regular addition is a typo. The output and input shapes of the adapter layers are equal and will depend on the location in the architecture where the adapter layers are added; for example, if the ith layer has an output shape of 32x32x15 and we add the adapter layer between ith and i+1th layer the adapter’s input and output shape would be 32x32x15, hence resulting in identity mapping of feature space during the initialization.

Reviewer comment (Experiments):
Both Figure 3 and Section 3.2 are conveying a very simple message: initialize the adapters with identity mapping. I would rather invest this space in more experiments with more insights (e.g. combining FEATHER with EATA).

Author response:
**Other suggestions:**

* Thanks for the suggestion about Figure 3 and section 3.2. We have updated the paper accordingly and have also added more experiments with FEATHER+EATA for comparison with EATA. Results are updated in Table 6.

Reviewer comment (Writing):
The proposed method is simple while the methodology section is not. I would try to simplify the writing of Section 3 and remove redundant paragraphs.

Author response:
**Other suggestions:**

* We have incorporated the writing and formatting suggestions in the updated version and have also adjusted the locations of the tables.

Reviewer comment (Presentation):
Please consider reorganizing the tables in page 7 such that each table is presented with its own paragraph.

Author response:
**Other suggestions:**

* We have incorporated the writing and formatting suggestions in the updated version and have also adjusted the locations of the tables.

Reviewer comment (Experiments):
For ImageNet-C experiments, please consider a similar setup to the CIFAR10/100-C experiments where the corruptions are ordered, similar to the continual evaluation in EATA.

Author response:
**Other suggestions:**

* In the updated version, we have included the results for ordered corruptions for ImageNet-C experiments (similar to EATA).
* The lifelong setting reported by EATA includes switching between the corruption and the source domain in an alternating fashion (please refer to https://github.com/mr-eggplant/EATA/blob/main/main.py#L123 for the official implementation), making it different from the followed lifelong definition of CTTA methods where the unseen domains (corruptions) are encountered in a continual fashion.  Nevertheless, for transparency, we report the FEATHER results with the EATA number taken from the paper directly (Table 11 in EATA paper). The comparison is presented below. Note that FEATHER shows an improvement over the average error rate, though the per corruption error rate reported by the EATA paper performs better, highlighting the overall effectiveness of FEATHER over EATA.


| Gauss. | Shot | Impul. | Defoc. | Glass | Motion | Zoom | Avg. |
|--------|------|--------|--------|-------|--------|------|------|
| $34.9_{ \pm 0.2}$ | $36.9_{ \pm 0.1}$ | $35.8_{ \pm 0.2}$ | $33.6_{ \pm 0.3}$ | $33.3_{ \pm 0.2}$ | $47.2_{ \pm 0.3}$ | $52.7_{ \pm 0.1}$ | $35.8_{ \pm 0.2}$ |
| $26.58_{ \pm 0.07}$ | $25.25_{ \pm 0.079}$ | $28.94_{ \pm 0.068}$ | $24.35_{ \pm 0.08}$ | $22.57_{ \pm 0.06}$ | $35.91_{ \pm 0.07}$ | $41.61_{ \pm 0.03}$ | $37.36_{ \pm 0.02}$ |

Reviewer comment (Experiments):
How to determine where to insert adapter? In the work, a combination of PWC and GCO servers as a basic adapter. And adapters will be inserted into the model between layers. However, there is no mention of where in the network the adapter should be inserted which is one of the most important aspects of methods based on adapters. Where to add, from shallow layer to deep layer, or from deep layer to shallow layer, there is no specific experiment to analyze.

Author response:
**Deciding where to add adapters:** Our approach based on adding adapters in between the layers is generic. Based on the computational budget of the deployment environment, the adapters can be added to the main architecture. In the updated version of the paper, we report the locations of the added adapter layers in Appendix F, and the configs for all the architectures will be released with the code base. The location of adapter parameters is dependent on the TTA environment. For example, if the expected noise is corruption, the domain shifts are expected in the initial layers, whereas if the domain shifts occur in semantic space, the suitable adapter locations would be in the deep(er) layers (close to the output layer). In the TTA setting, all the information about the domain shifts is usually not available a priori. To be generic and comply with the TTA setting, this decision should be made based on the dataset and the architecture for which FEATHER is being used. In our experiments and lifelong TTA settings, it would be unfair to predefine/suggest/assume any details related to the domain shifts. Hence the location of the used adapters is generic. (As described in Appendix F)

Reviewer comment (Theory):
Why ZERO AND IDENTITY INITIALIZATION preserves the source knowledge? Maybe a residual architecture with zero initialization will be more simple and effective?

Author response:
**Zero and Identity initialization vs residue architecture:** The zero and Identity initialization preserves source knowledge since it acts as an identity operation to begin with during the start. Even though a residue architecture with zero initialization is simpler, it is not as effective as our proposed PWC and GCO-based adapters. In our earlier experiments, we found that, when adding a residual layer, the network collapses when the domain changes occur in the continual setting and adapter parameters are updated. We speculate the primary reason to be the flow of signal to pass through the added parameters and, after the update, the network starts to collapse. We also tried some other initialization schemes like near zero init; however, those do not work well with the continual TTA setup.

Reviewer comment (Novelty):
Difference with ECoTTA [1]. In fact, adding adapters between layers has already been proposed in ECoTTA, and in their analysis experiments, they tried a similar variation of the method in their paper (refer to Architecture design in Section 4.2 in ECoTTA [1]). The choice and novelty of adapter structure is still open to question.

Author response:
**Difference with EcoTTA:** As compared to EcoTTA, FEATHER employs more sophisticated adapters, which is supported by better performance in continual TTA, shown in Table 4 (Table 7 in the previous version), where we compare with EcoTTA as well. Please refer to general comments for more discussion and comparison with EcoTTA. Also, one may refer to section 4.2 of the EcoTTA paper for architecture-specific details, which are quite different from FEATHER.

Reviewer comment (Evaluation):
What we should notice is that parameter efficiency does not mean resource efficiency. Inserting adapters into the original model will lead to more computational resource usage including FLOPs and GPU memory. But there was little or no performance improvement.

Author response:
**Savings in terms of computational/memory requirements:** Thanks for pointing this out and making this suggestion. We report the memory and compute savings in the general response and have updated the paper to also include these results. Our approach is more resource-efficient in terms of computational resources such as memory and FLOPs, which are usually required during the backward passes done during adaptation in test time. Further, we perform better than EcoTTA and other SOTA approaches in continual TTA, as highlighted in Table 4.

Reviewer comment (Evaluation):
Overclaim on the combination of other methods. Combined with TENT and CoTTA, there is no performance improvement.

Author response:
**Improvements when combining FEATHER with other methods:** We respectfully disagree that there is no improvement when FEATHER is combined with methods such as TENT and CoTTA. As compared to standard TENT and CoTTA, which require updating all the parameters (100%) during adaptation,  when these methods are augmented with FEATHER, significantly fewer parameters need to be updated, as shown in Table 6. Note that (as we mention in our common response as well as reported by the additional experiments) a reduction in trainable parameters directly affects the memory budget requirement as well as inference/adaptation time  (Table 7).

Reviewer comment (Reproducibility):
What is the objective function in the experiments of Table 1, 2, 3, 4, 5?

Author response:
**The objective function:** Thank you for pointing this out. The objective function in the experiments of Tables 1,2,3,4,5 is the same as that of CoTTA. We briefly mention this in the second line of Section 5.2. In the updated version of the paper, we highlight this in the table captions as well.

Reviewer comment (Theory):
The formulation of Lifelong/Continual Test-time adaptation has some inconformity with existing literature. In CoTTA [2], it is defined as adapting the model to the test samples and making predictions for them in an online manner. The description in Section 2 seems like an offline fashion.

Author response:
**Inconformity in the formulation of Lifelong/Continual TTA:** Note that, in Section 2, we start by defining the standard (non-continual) TTA setting. However, in the second last paragraph, we define the continual/lifelong TTA setting. Since the primary focus of this work is the behavior of adapters in a continual setting, we follow the continual setup for the entire experiments and results section and report the observed findings.

Reviewer comment (Experiments):
The proposed method is only applicable to CNNs.

Author response:
**Is FEATHER only applicable to CNNs?** As we mention in the common response, although we demonstrate our idea for generic CNN-based architectures for computer vision, our idea is general and can be applied to any deep learning architecture (including transformers) that allows the insertion of adapters between layers. Some implementational aspects will need to be changed based on the underlying architecture, but the rest of the idea is more broadly applicable.

Reviewer comment (Evaluation):
The authors argue for the importance of preserving the information of the source model in online TTA tasks. However, the proposed method does not exhibit outstanding performance compared to the CoTTA, which involves full fine-tuning of the model. Therefore, the authors fail to sufficiently explain why the use of adapters is suitable for online TTA tasks, apart from the fact that it reduces the number of trainable parameters.

Author response:
2. We would reiterate that our aim is not to surpass the performance of the full fine-tuning approach but rather to perform at least comparably with a significantly smaller number of parameters (which also translates into memory and time savings, as our results show). Our experimental results show that FEATHER is flexible in terms of the available computational budget and even outperforms CoTTA if the computational budget is increased (roughly 50% of CoTTA budget; please refer to Table 5). Please note the primary aim here is to reduce the computational budget. Moreover, the results in Table 3 for the ImageNet-to-ImageNetC dataset show that FEATHER achieves an improvement with only ~10% of parameters being trained.

Reviewer comment (Evaluation):
The authors claim that the proposed method excels in terms of parameter update costs; however, the cost of training the adapters inserted between the model's layers, in terms of memory and computation, is not significantly lighter compared to full fine-tuning.

Author response:
3. We respectfully disagree that a reduction in memory or computation cost does not align with the reduction in parameters (as shown in Table 7 and explained in the text around it, we get improvements on these metrics, too).  We would also like to highlight that **both the papers mentioned by you are about standard fine-tuning for transfer learning and not for TTA or lifelong TTA**, whereas our work is about lifelong TTA. Lifelong TTA presents a very different set of challenges than standard fine-tuning, which is done during the training phase, whereas TTA or lifelong TTA must perform fine-tuning at inference time using the unlabeled test input(s).

Reviewer comment (Experiments):
Is it possible to apply the proposed method to Transformers?

Author response:
**Is FEATHER only applicable to CNNs?** As we mention in the common response, although we demonstrate our idea for generic CNN-based architectures for computer vision, our idea is general and can be applied to any deep learning architecture (including transformers) that allows the insertion of adapters between layers. Some implementational aspects will need to be changed based on the underlying architecture, but the rest of the idea is more broadly applicable.

Reviewer comment (Evaluation):
Why does CoTTA generally outperform the proposed method?

Author response:
**Why CoTTA “outperforms” our method?** We respectfully disagree with this and would like to draw your attention to a few key points (also mentioned in our common response). Please note that, while CoTTA with **all** parameters trainable outperforms FEATHER marginally, CoTTA also has significantly more trainable parameters. In contrast, FEATHER only uses ~7.2% to 12.2% trainable parameters and yet performs comparably to CoTTA. Further, we demonstrate in Table 5 that with only 50.32% trainable parameters, FEATHER **outperforms** CoTTA, which highlights that even adapting just around half the number of parameters is enough.