Controlling Forgetting with Test-Time Data in Continual Learning

## Abstract

Abstract Foundational vision-language models have shown impressive performance on various downstream tasks. Yet, there is still a pressing need to update these models later as new tasks or domains become available. Ongoing Continual Learning (CL) research provides techniques to overcome catastrophic forgetting of previous information when new knowledge is acquired. To date, CL techniques focus only on the supervised training sessions. This results in significant forgetting yielding inferior performance to even the prior model zero shot performance. In this work, we argue that test-time data hold great information that can be leveraged in a self supervised manner to refresh the model’s memory of previous learned tasks and hence greatly reduce forgetting at no extra labelling cost. We study how unsupervised data can be employed online to improve models’ performance on prior tasks upon encountering representative samples. We propose a simple yet effective student-teacher model with gradient based sparse parameters updates and show significant performance improvements and reduction in forgetting, which could alleviate the role of an offline episodic memory/experience replay buffer.

## 1 Introduction

Foundation models in computer vision have shown impressive performance on various down stream tasks and domains which renders them a key building block of various solutions including generative vision language models [, , ].
In spite of these models’ generality, carefully fine-tuning them on specific tasks and domains usually results in significant performance gains.
However, naively adapting pretrained models to changes in data distribution or new tasks faces the well-known catastrophic forgetting phenomena [] where new learning sessions interfere with what a model has previously acquired. To overcome catastrophic forgetting, Continual Learning (CL) has emerged as a branch of machine learning to enable models to continuously adapt to evolving distributions of training samples or supervision signals over time.
A variety of approaches have been proposed to mitigate catastrophic forgetting, such as regularization-based methods [, , ], external memory approaches [, ], and dynamic model architecture techniques []. Most of these works typically focus on models trained from scratch and might fail when applied to large pretrained models.
The rise of large foundation models has sparked increased interest in merging CL with the benefits offered by potent pre-trained models [, , , , , ].

Despite the increased attempts to efficiently improve foundational models performance on new streams of data [, , , , , , , , , , ], forgetting is still a significant problem in applications of continual learning[, ].

Importantly continual learning systems are often deployed throughout their lifecycle, performing inference on large amounts of unsupervised data. We argue that an important factor is to continuously learn irrespective of whether supervision is provided or not. Despite the promise of continual learning, most works focus solely on training in distinct supervised sessions, while the model remain passive and frozen at test-time.

Consider an embodied agent equipped with a Vision Language model (VLM) and can recognize various objects in its environment, and answers users queries, upon introducing new types of objects, layouts, or skills, it is still expected to encounter instances of previously learned objects or tasks at evaluation time. We suggest that a main ingredient of overcoming catastrophic forgetting and the successful accumulation of information lies in leveraging test-time data to refresh the model knowledge on previously learned tasks. Furthermore, the data encountered during test time represents the distribution of interest that directly impacts the agent’s tasks. We propose that data learned in the past but never encountered during test time is of lesser importance and can indeed be forgotten to enhance performance on data frequently encountered during deployment.

We thus consider a scenario where a model is continually trained on supervised datasets, while in-between receiving the supervised datasets, unsupervised data becomes available during the model deployment that can be used for controlling forgetting. In this work we constrain the unsupervised adaptation to be online, to allow a practical computational overhead. In particular data privacy constraints with test data encountered in deployment are often more rigid [] necessitating online algorithms for this phase that discard samples after they are processed.

Figure: Figure 1: An illustration of our proposed Continual Learning with Interleaved Test Time Learning. Following each session of supervised learning, the model is deployed to adapt in an unsupervised setting. It can encounter data distributions encompassing all previously encountered tasks or sessions. The model adapts on the classes of the current task, while trying to minimize the forgetting on all the classes of previously seen tasks.
Refer to caption: extracted/5677516/imgs/seting.png

Test-Time Adaptation (TTA) [] and Continual Test-Time Adaptation (CoTTA) [] are related research areas that focus on leveraging test-time data for dynamic model adaptation. These areas focus on adapting the model towards unknown distribution shifts using test-time data, while our formulation aims to use test-time data to control the model forgetting, without any assumption of distribution shifts from training to test data.

Figure: Figure 2: An illustration of our method DoSAPP. It utilises teacher-student($\mathcal{M}_{T},\;\mathcal{M}_{S}$) models respectively. During Supervised Continual Learning phase, $\mathcal{M}_{S}$ undergoes sparse parameter selection through a gradient based scoring function $\mathcal{F}$, followed by supervised training of these selected parameters $\theta^{\textbf{m}}\in\theta^{S}$. After each gradient update step, $\mathcal{M}_{T}$ parameters, $\theta^{T}$, are updated through a weighted exponential smoothing based on affine projection of the boolean mask: m. The affine projections are controlled through dual momentum terms $\delta,\gamma$ for $\mathcal{M}_{T}$ and $\mathcal{M}_{S}$ respectively. Now both $\mathcal{M}_{T},\;\mathcal{M}_{S}$ are deployed for the unsupervised test time learning where the $\mathcal{M}_{S}$ is adapted based on the "pseudo label groundtruth" generated from $\mathcal{M}_{T}$ - $\mathcal{M}_{S}$ logits comparison. Finally $\mathcal{M}_{T}$ model again undergoes weighted smoothing, with dual momentum terms $\delta,\lambda$ for $\mathcal{M}_{T}$ and $\mathcal{M}_{S}$ model respectively such that $\gamma<\lambda<\delta$. This 2 phase approach preserves the generalizations over previous knowledge along with adaptibility on latest task.
Refer to caption: x1.png

To the best of our knowledge, we are the first to explore how test-time data can be leveraged in a continual learning setting to reduce forgetting. We consider the foundation model CLIP [] for our experiments since it has been shown to encompass an extensive knowledge
base and offer remarkable transferability[, ]. It undergoes through supervised and unsupervised sessions, leveraging the unsupervised data to control forgetting.

We propose an effective approach based on student-teacher models with sparse parameters selection based on gradient values. Student and teacher models suggest labels for test-data and the predictions from the most confident model is used to update the student model, where the teacher is updated in an exponential moving average adding a stability component to the learning process. We show that such a simple approach achieves significant improvements on all studied sequences. Our approach is stable in class incremental learning(CIL) especially in the challenging setting where no replay buffers are used, which in many cases can be a critical bottleneck.

Our contributions are as follows: 1) We propose a new setting for continual learning where test-time data can be leveraged especially in the challenging scenario of CIL-CL. 2) We investigate different baselines for this setting. 3) We propose a novel approach that illustrates the utility of test-time data in supervised continual learning and the significant reduction in forgetting without any external replay buffer.

In the following we discussed the closely related work, Section 2 and present our setting, and our approach, Section 3 we evaluate our approach on various CL sequences, Section 4, perform ablations on different components of our approach, Section 5, put forth some limitations of our work, Section 6 and conclude in Section 7.

## 2 Related Work

Continual Learning considers learning in an incremental manner where training data is received at various time-steps (sessions). The typical problem is catastrophic forgetting [] of previously learned information. We refer to [] for a survey on class incremental learning where different classes are learned at distinct sessions, a setting we consider in this work. Weight regularization methods [, ] and functional regularization [, ] direct the training to stay optimal for tasks of previous sessions via various regularization terms.
Experience Replay[] is usually deployed where samples of previous training sessions data are replayed during new sessions to reduce forgetting. In this work, we consider continual learning with limited or no replay. Our work is orthogonal to other continual learning methods and can be combined with any CL method in the supervised training sessions.

Continual Learning from Pre-trained Models Due to the abundance of powerful pre-trained models [, , ] continual learning that begins with a pre-trained model is becoming a popular paradigm. Recent methods [, ] have utilised a Teacher-Student framework for knowledge distillation on previous seen tasks. But, these methods utilise an additional buffer to mitigate catastrophic forgetting. This often entails significant memory [, ]. Additionally, such methods often face an outdated logit problem, as the memory-stored logits are not updated to preserve information on previous tasks. Boschini et al. [] addresses this issue by updating logits stored in the past using task boundary information (e.g., input’s task identity) during training, but it may not always be available, especially in task-free CL setups. However, foundation models [, ] often have a reasonable initial performance on novel tasks, indicating some pre-existing knowledge relevant to these tasks. Zhang et al. [] utilises this property and preservers generic knowledge by modifying only a small set of parameters based on gradient scoring mechanism. But this method also suffers from recency bias since the gradient scores are computed for current task and only those sparse parameters are updated based on current task scores. Moreover, none of the methods utilise test data in continual learning scenario, and leave a strong potential for self supervised techniques to capture robust feature representations.

Test-Time Adaptation where a pre-trained model is adapted based on the test data has been heavily studied in recent years. Typically the goal is to improve performance on the test data being used for adaptation itself, while we focus on using this data to control forgetting of past tasks. A number of methods have been proposed for TTA including those that leverage self-supervised learning [], batch normalization [, ], entropy minimization [, ], as well as pseudo labeling [, ].

Continual Test-Time Adaptation Recent work has studied the setting of performing Online Test-Time Adaptation where the distribution of test-time data is changing over time []. This is distinct from the proposed setting as we focus on the setting where the model is updated with supervised data, while the test-time data is leveraged to control forgetting on the supervised tasks.

## 3 Methodology

### 3.1 Setting

We consider a setting where a sequence of supervised datasets $[{\mathcal{D}_{1}^{s},\mathcal{D}_{2}^{s},.....\mathcal{D}_{T}^{s}}]$ drawn from different distributions are observed at incremental training sessions $t$ ranging from 0 0 to $T$, where $\mathcal{D}_{t}^{s}={(\mathbf{x}^{t}_{i},y^{t}_{i})}^{N_{t}}_{i=1}$ is the $t$ incremental session with $N_{t}$ instances. Here the training instance $\mathbf{x}_{i}^{t}\in\mathbb{R}^{D}$ belongs to class $y_{i}\in Y_{t}$, where $Y_{t}$ is the label space of task/dataset at $t$ step. $Y_{t}\cap Y_{t^{\prime}}=\phi$ for $t\neq t^{\prime}$, where $t^{{}^{\prime}}$ is any other training session. During a given training session $t$ data samples only from $\mathcal{D}^{s}_{t}$ can be accessed. The aim of CIL is to progressively build a unified model encompassing all previously encountered classes. This involves gaining insights from new classes while retaining knowledge from previous ones. Model’s performance is evaluated over all the seen classes $\mathcal{Y}_{t}=Y_{1}\cup\cdots Y_{t}$ after each incremental task/dataset. Formally, the target is to fit a model $\mathcal{M}(\mathbf{x};\boldsymbol{\theta}):X\rightarrow\mathcal{Y}_{t}$ that achieves a minimal loss $\mathcal{L}$ across all testing datasets $\mathcal{D}^{e}_{t}$ :

$$ $\sum_{\left(\mathbf{x}_{j},y_{j}\right)\in\mathcal{D}^{e}_{1}\cup\cdots \mathcal{D}^{e}_{T}}\mathcal{L}\left(\mathcal{M}\left(\mathbf{x}_{j}; \boldsymbol{\theta}\right),y_{j}\right)$ (1) $$

where $\mathcal{L(.,.)}$ measures the difference between prediction and groundtruth label. $\mathcal{D}^{e}_{t}$ denotes a testing set of task $t$. Finally $\theta$ denotes the model parameters.

After training is complete on each $\mathcal{D}_{t}^{s}$ the model is put into production until $\mathcal{D}_{t+1}^{s}$ becomes available for supervised training. Between supervised phases an unsupervised dataset, $\mathcal{D}_{t}^{u}$, is observed corresponding to test-time data encountered in production. Note that this unsupervised data can be drawn from a different distribution than the supervised data, including the distributions of old supervised datasets/tasks. Our goal is to leverage this data to control forgetting of the model by allowing online unsupervised adaptation. Figure 1 depicts our setting. Note that we evaluate our models on test datasets $\{\mathcal{D}^{e}\}$ that are distinct in terms of instances from those used during the self-supervised online adaptation phase to adequately measure models generalization.

We further note that although supervised phases may permit multiple passes through the data until convergence, it would be impractical to collect unsupervised data in production and then perform adaptation on it, we thus restrict the unsupervised phase to be in the online setting [, , ]. This is especially important in cases were data privacy is important e.g., assistant robot in a private smart home environment.

### 3.2 DoSAPP: Do uble S moothing via A ffine P rojected P arameters

Figure: Algorithm 1 DoSAPP algorithm for continual and test time learning

We propose a simple yet effective method for continual test-time learning, Double Smoothing via Affine Projected Parameters aka DoSAPP. Our approach combines two key components:
1) sparse and local updates: to reduce forgetting, maintain generalization, and ensure efficient updates, and 2)
teacher-student framework to promote stability in online updates and minimizes forgetting.
In the continual test time learning we can identify two distinct phases of learning as outlined in the following.

### Phase 1: Continual Learning Supervised Training with Sparse Selected Parameters

Our primary objective is to swiftly accumulate new knowledge without catastrophically forgetting the generic knowledge both at training and test-time. To achieve this, we opt for updating only a small subset of selected parameters. It has been suggested by Zhang et al. [] that for a generic prertrained model like CLIP and a given task, relevant parameters can be identified before training, and updating only those parameters would result in a reduced forgetting of previous knowledge.
Further Geva et al. []suggested that MLP blocks in a transformer model emulate key-value neural memories, where the first layer of MLP acts as memory keys operating as pattern detectors. This suggests that for updating knowledge of previously known ”patterns”, it might be sufficient to update only first MLP layer parameters. Thus we limit candidate parameters to the first MLP layer parameters of each transformer block in the CLIP model []. From these candidate parameters of the first MLP layer of each transformer, we select top-K(K=$c$) parameters. This results in efficient training without loss of previously acquired knowledge as all other layers remain frozen.

Following [] we use the gradient magnitude of the loss w.r.t. the incoming data as a score of how relevant a parameter is, the larger the gradient magnitude the larger the expected decrease in loss after small changes to that parameter.
We refer to the model being optimized as $\mathcal{M}_{S}$. Upon recieving supervised data, we first estimate the most relevant parameters, $\boldsymbol{\theta}^{\textbf{m}}$ such that ($\boldsymbol{\theta}^{\textbf{m}}\in\boldsymbol{\theta}^{S}$).

$$ $\mathcal{F}\left(\boldsymbol{\theta}_{ij}^{S},\mathcal{D}^{s}_{t}\right)=\left \|\frac{1}{N_{t}^{\prime}}\sum_{k=1}^{N_{t}^{\prime}}g_{ij}\left(x_{k}\right) \right\|,$ (2) $$

where $g_{ij}\left(x_{k}\right)$ is the gradient of the loss function($\mathcal{L}(\mathcal{M}_{S},x_{k},y_{k})$) regarding the parameter $\boldsymbol{\theta}_{ij}^{S}$ evaluated at the data point and its label $x_{k},y_{k}\in\mathcal{D}^{s}_{t}$. The loss function $\mathcal{L}(\mathcal{M}_{S},x_{k},y_{k})$ is the same CLIP loss, and the entire data is iterated once to compute the gradient score as given in Eq 2. Specifying the sparsity threshold($c$), top-K(K=$c$) most relevant parameters are selected. We set $c=0.1$ as shown in []. This results in a binary mask m where only selected parameters are updated and others are masked out and kept frozen.

#### Teacher Student Framework

To insure stability later during online updates and reduce forgetting, we utilise a Student-Teacher framework [, , ] where the student model is denoted by $\mathcal{M}_{S}(\boldsymbol{\theta}^{S})$ and the teacher model is denoted by $\mathcal{M}_{T}(\boldsymbol{\theta}^{T})$.

During both train and test time, teacher model $\mathcal{M}_{T}$ parameters $\boldsymbol{\theta}^{T}$ move with exponentially moving average (EMA) of student model parameters $\boldsymbol{\theta}^{S}$. Normally in a teacher student framework, all teacher model parameters move similarly towards the student parameters with a single smoothing parameter (momentum). However, in tables 1 and 4 we show that a single smoothing parameter is insufficient and yields poor performance. Indeed, in our case most of student model parameters remain frozen and only a small portion is updated, we propose that teacher model’s parameters corresponding to the student frozen parameters should move at a different pace than those selected for updates. Therefore we use dual smoothing parameters (referred as momentum parameters) based on affine transformation of the binary mask m to adapt the teacher parameters $\boldsymbol{\theta}^{T}$.

#### Weighted exponential smoothing with dual momentum

After each gradient update step($i$) for $\mathcal{M}_{S}$, parameters of $\mathcal{M}_{T}$ are updated by EMA of the student model parameters. Typically, EMA is governed by

$$ $\boldsymbol{\theta}^{T}_{i+1}=\delta\boldsymbol{\theta}^{T}_{i}+(1-\delta) \boldsymbol{\theta}^{S}_{i+1}$ (3) $$

where $\delta$ is the smoothing parameter. Further it has been shown in ([, , ]) that setting $\delta$ to a high value(eg 0.9999), maintains a stable teacher model that can be considered as a strong reference for past tasks $\{0,\dots,t-1\}$. But updating the teacher model with single smoothing parameter in case where parameters are masked creates a dissonance and increases forgetting because all the parameters are updated with equal importance, disregarding those parameters which are selected by the gradient scoring function(where $[\textbf{m}_{ij}=1]$). To account for masking, we modify Eq 3 as

$$ $\boldsymbol{\theta}^{T}_{i+1}=\boldsymbol{p}\boldsymbol{\theta}^{T}_{i}+ \boldsymbol{q}\boldsymbol{\theta}^{S}_{i+1}$ (4) $$

where $\boldsymbol{p}$ and $\boldsymbol{q}$ denote the smoothing parameters for the teacher and student model respectively, and can be computed as

$$ $\displaystyle\boldsymbol{p}=(\gamma-\delta)\textbf{m}+\delta$ (5) $\displaystyle\boldsymbol{q}=(\delta-\gamma)\textbf{m}+1-\delta$ $$

where $\gamma<\delta$. This means that the selected parameters of the teacher model ([$\textbf{m}_{ij}=1$]), move little bit faster towards the student model as compared to the frozen parameters(where [$\textbf{m}_{ij}=0$]). As such, parameters where [$\textbf{m}_{ij}=0$] will move at a slow rate of $\delta$ and unmasked parameters would be updated with $\gamma$. When $\gamma=\delta$, the weighted scheme becomes EMA with single smoothing parameter. A detailed proof is given in appendix A.1.

### Phase 2: Unsupervised Test Time Learning(TTL)

After supervised training is completed, both $\mathcal{M}_{T}$ and $\mathcal{M}_{S}$ are deployed for Test Time Learning(TTL). We consider teacher($\mathcal{M}_{T}$) and student($\mathcal{M}_{S}$) models as two experts on different data distributions, the $\mathcal{M}_{S}$ on the most recent and the $\mathcal{M}_{T}$ on previous sessions distributions.

We take inspiration from Out Of Distribution (ODD) literature [], where a sample has to be identified as In Distribution (ID) for a given predictor with a score function predicting high values for ID samples as opposed to OOD samples. Recently it has been shown that using the un-normalized maximum logit output of a given predictor as an ID score is significantly more robust than softmax probability []. Indeed the softmax probability is shown to provide high probability predictions even for unknown samples [], which we want to avoid in our case. Note that for CLIP the logit corresponds to the cosine similarity of image batch with given text features.

Following [], we use maximum logit value of each expert as an ID score, and select for each test sample the expert with the highest ID score indicating that the sample is likely to be better represented by said expert. We then accept the pseudo label of the selected expert. Formally the pseudo label can be calculated as follows:

$$ $\hat{y}=\begin{cases}\hat{y_{T}}&\text{ if }l_{T}\geqslant l_{S}\\ \hat{y_{S}}&\text{ otherwise. }\end{cases}$ (6) $$

where $\hat{y}$ is the accepted pseudo label and $l_{T}=\max(\mathcal{M}_{T}(\textbf{x}))$ and $l_{S}=\max(\mathcal{M}_{S}(\textbf{x}))$ are the maximum logit score for teacher and student model respectively,

**Table 1: Effect of Momentum($\gamma,\lambda$) on Average Accuracy(Acc in % ), Average Forgetting(F.) and First Task Accuracy(FTA.) *0.9999, 0.8, 0.9 have been used in the main results.**
| Momentum($\gamma$, $\lambda$) | Aircraft |  |  |
| --- | --- | --- | --- |
| Acc. ($\uparrow$) | F.($\downarrow$) | FTA. ($\uparrow$) |  |
| 0.9999, 0.9999 | 23.99 | 18.36 | 12.15 |
| 0.5, 0.9 | 38.41 | 3.27 | 37.64 |
| 0.7, 0.9 | 37.22 | 3.05 | 37.72 |
| 0.8, 0.9* | 39.40 | 2.61 | 38.13 |
| 0.8, 0.6 | 37.06 | 5.12 | 29.63 |
| 0.8, 0.5 | 32.95 | 3.40 | 26.33 |

and similarly $\hat{y_{T}}=\arg\max(\mathcal{M}_{T}(\textbf{x}))$ and $\hat{y_{S}}=\arg\max(\mathcal{M}_{S}(\textbf{x}))$ are the pseudo labels by teacher and student models respectively. During test-time training the student model $\mathcal{M}_{S}$ is updated by minimising CLIP contrastive loss given pseudo label $\hat{y}$. In realistic settings, often multiple iterations on test data is not always possible, for eg, a streaming data pipeline. We too mimic this setting, where the entire data is processed only once during TTL phase.

Similar to the above mentioned supervised phase, we also here apply sparse local updates to $\mathcal{M}_{S}$. However, estimation of masks based on the online data might be noisy, and largely reduce the efficiency as gradients of all parameters must be estimated for each mini batch of test samples. To overcome this, and following the assumption that test data are drawn from the distributions of all previous tasks, we leverage the masks estimated for previous tasks.
We accumulate a union of the binary masks ($\textbf{m}_{u}$) over all the previously seen tasks $t$ such that $\textbf{m}_{u}=\textbf{m}_{1}\cup\textbf{m}_{2}\cup......\textbf{m}_{t}$. To maintain the same sparsity level ($c=0.1$) of performed updates, we further select the same top-K (K=$c$) most relevant parameters, from these new masked $\textbf{m}_{u}$ parameters based on their previously computed gradient scores.

Finally $\mathcal{M}_{T}(\boldsymbol{\theta}^{T})$ is updated using the same dual momentum scheme, but with different smoothing vectors $\boldsymbol{p}^{\prime},\boldsymbol{q}^{\prime}$ as:

$$ $\boldsymbol{\theta}^{T}_{i+1}=\boldsymbol{p}^{\prime}\boldsymbol{\theta}^{T}_{ i}+\boldsymbol{q}^{\prime}\boldsymbol{\theta}^{S}_{i+1}$ (7) $$

where $\boldsymbol{p}^{\prime}=(\lambda-\delta)\textbf{m}+\delta$ and $\boldsymbol{q}^{\prime}=(\delta-\lambda)\textbf{m}+1-\delta$. In TTL phase, the momentum parameter $\lambda$ is kept such that $\gamma<\lambda<\delta$. This means that $\boldsymbol{\theta}^{T}$ moves more slowly in direction of $\boldsymbol{\theta}^{S}$ during TTL phase as compared to the supervised phase. As we encounter frequent, and possibly noisy, online updates, stability is better insured by a slower pace of movements towards student parameters. We show the sensitivity of our method on choice of momentum values $\lambda,\delta$ in table 1. A high $\delta$ has been chosen to keep the Teacher model stable as shown in [, , ]. It can be clearly seen that when $\gamma=\lambda$(single momentum EMA), the performance significantly drops. DoSAPP is less sensitive to on chocie of $\gamma$, but it highly depends on $\lambda$. We can also see that as $\lambda<\gamma$, the performance again drops.
The algorithm can be fully understood as given in 1

## 4 Experiments

### 4.1 Setup

Architecture: We apply DoSAPP to vision-language classification tasks, given their relatively robust knowledge measurement in such tasks. CLIP-ViT/B-16 [], is used as backbone. We report the accuracies recorded by the Teacher model. We refer to [] for hyperparameters selection other than dual momentums, which are given in Appendix A.2.

Datasets: We consider five different vision datasets, three fine-grained(Aircraft[], CUB[], Stanford Cars[], Oxford Pets[], one coarse dataset(CIFAR100 []) and one out-of-distribution dataset(GSTRB[]). These datasets are chosen primarily based on their initially low zero-shot performance with CLIP pre-trained models. To form the
continual learning sequences, we split each dataset into 10 subsets with disjoint classes composing 10 tasks. For all the datasets, the training data is used in supervised learning phase. The test data is divided into 2 splits, namely $\mathcal{D}^{u},\mathcal{D}^{e}$ where $\mathcal{D}^{u}$ is utilised for test-time unsupervised learning and $\mathcal{D}^{e}$ is used for evaluation.

Evaluation Metrics: After each supervised session $t_{i}$ and the following test-time adaptation session, we evaluate the model test performance on holdout datasets from all $T$ tasks. In order to do this, we construct the matrix $R\in\mathbb{R}^{TxT}$, where $R_{i,j}$ is the test classification accuracy of the model on task $t_{j}$ after observing the last sample from task $t_{i}$. Thus, we compute Average Accuracy(Acc. = $\frac{1}{T}\sum_{i=1}^{T}R_{T,i}.$ ) and Average Forgetting(F. = $=-\frac{1}{T-1}\sum_{i=1}^{T-1}R_{T,i}-R_{i,i}.$) []. Taken together, these two metrics allow us to assess how well a continual learner solves a classification problem while overcoming forgetting. All experiments have been done on NVIDIA A100 GPU and each one takes approximately 1 hour for completion.

**Table 2: Acc(Average Accuracy) % ($\uparrow$) and F. (Forgetting)$\downarrow$ of different methods using CLIP with trainable vision and text encoders, without any Replay Buffer. DoSAPP can achieve positive backward transfer - forgetting is negative on Cars data. All experiments are mean of 5 experiments with random seeds. std. is not shown for ease of reading and space constraint.**
| Method | Aircraft | Cars | CIFAR100 | CUB | GTSRB |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) |  |
| CLIP-Zeroshot [] | 24.45 | - | 64.63 | - | 68.25 | - | 55.13 | - | 43.38 | - |
| Finetune [] | 18.63 | 39.93 | 51.64 | 25.65 | 46.26 | 37.78 | 45.74 | 26.62 | 21.76 | 55.48 |
| SL | 10.81 | 50.81 | 23.49 | 30.42 | 38.03 | 42.67 | 28.60 | 33.82 | 5.14 | 62.31 |
| MAS [] | 33.69 | 27.50 | 69.43 | 9.18 | 63.88 | 21.16 | 61.72 | 12.05 | 42.04 | 25.38 |
| L2P[] | 32.20 | 21.73 | 67.04 | 11.22 | 67.71 | 18.81 | 64.04 | 6.82 | 75.45 | 2.68 |
| DualPrompt [] | 26.61 | 17.20 | 63.30 | 18.67 | 61.72 | 19.87 | 64.38 | 12.94 | 69.65 | 8.43 |
| SLCA [] | 29.40 | 11.45 | 62.65 | 4.42 | 70.03 | 0.19 | 53.87 | 7.75 | 46.01 | 0.83 |
| ZSCL [] | 30.96 | 15.65 | 67.79 | 8.27 | 80.50 | 1.05 | 61.09 | 7.69 | 62.92 | 13.54 |
| SparseCL [] | 31.95 | 19.77 | 71.57 | 5.38 | 69.35 | 15.23 | 62.50 | 9.66 | 48.99 | 24.91 |
| SPU [] | 30.94 | 28.36 | 69.41 | 16.91 | 58.80 | 26.37 | 62.31 | 7.2 | 43.06 | 19.16 |
| DoSAPP | 39.14 | 12.55 | 74.87 | -0.74 | 79.16 | 7.73 | 68.17 | 2.15 | 72.33 | 1.02 |
| ER methods |  |  |  |  |  |  |  |  |  |  |
| ER[] | 41.42 | 31.38 | 69.08 | 16.42 | 82.86 | 3.41 | 64.07 | 17.72 | 96.28 | -7.48 |
| ER + LWF[] | 36.08 | 18.12 | 72.56 | 4.04 | 74.32 | 8.16 | 65.11 | 5.90 | 53.56 | 11.86 |
| ER + PRD [] | 37.11 | 17.35 | 74.08 | 3.75 | 79.66 | 3.10 | 65.92 | 6.55 | 63.00 | 12.44 |
| SPU + ER=1000 | 44.43 | 14.42 | 77.51 | 3.26 | 83.99 | -0.39 | 71.51 | 4.84 | 94.25 | -7.87 |
| DoSAPP + ER=200 | 47.32 | 8.10 | 79.17 | 3.92 | 88.41 | -1.96 | 74.39 | 2.77 | 83.67 | 1.92 |

### 4.2 Results

We compare variety of baselines with our proposed method in table. 2. Along with the methods mentioned in table. 2, we also compare our method with self labeling(SL) where the groundtruth pseudo label comes from the trained model itself(without any student-teacher framework). When comparing methods without ER, DoSAPP achieves state of the art results in all the five datasets used in the experiments. This highlights the fact that test time data can be utilised for improving transferability as well as preserving previously learned knowledge. Even when comparing methods with ER, DoSAPP(without ER) gives a comparable performance in almost all the datasets. We note that SPU+ER employs a very high buffer of 1000, which attributes to such a high performance in some datasets like Cifar100 and GTSRB. Although our method is robust enough to be used without ER and our primary motivation is to circumvent the usage of buffer, but we still present results with a small buffer(DoSAPP+ER, ER=200), for a comparison to the baselines using ER. DoSAPP + ER outperforms all other baselines except on GTSRB by significant margin.

### 4.3 Class Incremental Long Sequence scenario with domain shift

We also consider the case where we have a long sequence of tasks each to be trained in a class incremental fashion. For these experiments, we combined the 10 tasks of Aircraft data[] and 10 tasks of Cars data[]. This firstly creates a long sequence of tasks in a class incremental scenario, and secondly causes a domain shift after 10 tasks of aircraft. From table 3, it can be clearly seen that our proposed method DoSAPP outperforms SPU without ER and Finetune(without any TTL phase). Further it can be inferred that in other baselines, there is a recency bias towards the current task, whereas in DoSAPP, with a marginal decrease of $3.8\%$ on current task accuracy(CTA), there is an overall increase in the average accuracy and the first task accuracy. This shows that our approach retains the knowledge on first task as well adapts well on the current task, with strong generalisation performance.

**Table 3: Averge Accuracy(Avg Acc.), First Task Accuracy(FTA) and Current Task Accuracy(CTA) measured for long sequnce of tasks from concatenation of aircraft [] and cars [] dataset. All experiments are mean of 5 randomised experiments with different seeds.**
| Method(CLIP) | Avg Acc | FTA | CTA |
| --- | --- | --- | --- |
| Finetune(no TTL) | 35.24 $\pm$ 0.87 | 5.90 $\pm$ 1.20 | 75.44 $\pm$ 0.52 |
| SPU | 39.62 $\pm$ 1.62 | 24.31 $\pm$ 0.30 | 74.94 $\pm$ 2.43 |
| DoSAPP | 45.01 $\pm$ 0.31 | 30.63 $\pm$ 0. 76 | 71.13$\pm$ 1.17 |

## 5 Ablation Study

In this section we quantitatively analyse the effect of different components of our proposed method DoSAPP. We evaluate the effects of each component in an incremental fashion as seen in table. 4. Starting with only a student and teacher model setup only, we subject it to TTL data and this forms our baseline. Next we compare with localised sparse updates for the first MLP layer of each of the transformer blocks. This gives an increase in performance in 4 out of 5 datasets. It is to be noted that the momentum used to update the teacher model is according to Eq. 3. We then take the union of supervised task masks to use them at TTL phase, but this deteriorates performance since the masked parameters and unmasked parameters are updated with a single momentum. Finally we add our dual momentum approach which gives best performance. We also subject our approach to a more challenging scenario where the tasks in TTL phases are class-imbalanced. Here we sample each task from a symmetric Dirichlet distribution whose concentration parameter is the length of each task. This causes a high imbalance of classes within each task, and sometimes, even absence of certain classes. This imbalanced case is of particular importance since in real settings, test suites are often skewed.This is done by randomly sampling classes from a Dirichlet’s distribution. Although the performance is inferior to the balanced case, it should not be interpreted as a drawback. This is because, the model should adapt more to the classes that are seen often in TTL phases and loss of performance on rarely seen classes is but natural.

**Table 4: Acc(Average Accuracy) % ($\uparrow$) and F. (Forgetting)$\downarrow$ of different components of DoSAPP. All the experiments are averaged over 5 randomised trials with different seeds.**
| Components of DoSAPP | Aircraft | Cars | CIFAR100 | CUB | GTSRB |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) | Acc. ($\uparrow$) | F.($\downarrow$) |  |
| Only Teacher-Student | 30.12 | 3.50 | 67.72 | 3.66 | 77.82 | 5.17 | 62.67 | 4.11 | 53.57 | 5.38 |
| + sparse params | 34.16 | 8.61 | 69.42 | 3.41 | 71.93 | 8.24 | 66.32 | 3.98 | 55.32 | 5.81 |
| + union of mask | 32.99 | 10.36 | 67.48 | 6.91 | 76.77 | 10.11 | 65.70 | 26.62 | 56.82 | 3.21 |
| +dual momentum* | 39.14 | 2.55 | 74.87 | -0.74 | 79.16 | 7.73 | 68.17 | 2.15 | 72.33 | 1.02 |
| + imbalanced TTL | 35.99 | 5.22 | 72.68 | 6.38 | 75.70 | 9.81 | 64.84 | 3.73 | 68.17 | 5.63 |

## 6 Limitation

DoSAPP is a robust algorithm which can be potentially applied to any CL technique for unsupervised adaptation of Test Time Data. However, since it utilises the test data, its primary bottleneck becomes the quality of test data especially if its highyly skewed. Another limitation is the increase in the computational budget due to two deployed models: Student-Teacher framework. We address this by leveraging efficient sparse parameter selection method.

## 7 Discussion and Conclusion

In this work, we discuss how to leverage test-time data to improve models’ representation of previous tasks, mimicking human learning and striving for real intelligent agents. In summary, to the best of our knowledge we are the first to explore test-time learning to control forgetting. We show that test-time data can provide a great source of information when leveraged correctly. Our method, DoSAPP, was able to significantly improve over the zero-shot performance of CLIP when continually learning a dataset without any replay and with no specific CL method applied at the supervised training session. DoSAPP is stable due to sparse parameter updates and the weighted EMA teacher-student framework. Further during TTL, the max-logit in distribution scores makes it more robust to class imbalance than other strategies.

## 8 Acknowledgements

E.B. and V.S. acknowledge funding from NSERC Discovery
Grant RGPIN- 2021-04104. This research was enabled in part by compute resources provided by Digital Research Alliance of Canada (the Alliance) and Calcul Québec.

## Appendix A Appendix / supplemental material

### A.1 Derivation for dual momentum

In section 3, the teacher model parameters $\boldsymbol{\theta}^{T}_{i}$ undergo exponential moving average as

$$ $\boldsymbol{\theta}^{T}_{i+1}=\boldsymbol{p}\boldsymbol{\theta}^{T}_{i}+ \boldsymbol{q}\boldsymbol{\theta}^{S}_{i+1}$ (8) $$

where $\boldsymbol{p}$ and $\boldsymbol{q}$ denote the smoothing parameters for the teacher and student model respectively, and can be computed as

$$ $\displaystyle\boldsymbol{p}=\alpha_{1}\textbf{m}+\beta_{1}$ (9) $\displaystyle\boldsymbol{p}=\alpha_{2}\textbf{m}+\beta_{2}$ $$

where $\alpha_{i}$ and $\beta_{i}$ for $i\in\{1,2\}$ are the coefficients for affine transformation of the boolean mask vector m.

To account for masked parameters, two momentum values $\delta,\gamma$ are introduced for teacher and student models respectively, such that for the teacher model, affine coefficients $\alpha_{1},\;\beta_{1}$ are computed by solving the equations:

$$ $\displaystyle\alpha_{1}[\textbf{m}_{ij}=1]+\beta_{1}=\gamma\;,\quad\quad\quad \alpha_{1}[\textbf{m}_{ij}=0]+\beta_{1}=\delta$ (10) $$

and $\alpha_{2},\beta_{2}$ are computed by solving the equations

$$ $\displaystyle\alpha_{2}[\textbf{m}_{ij}=1]+\beta_{2}=1-\gamma\;,\quad\quad \quad\alpha_{2}[\textbf{m}_{ij}=0]+\beta_{2}=1-\delta$ (11) $$

This gives

$$ $\displaystyle\alpha_{1}=\gamma-\delta,\quad\beta_{1}=\delta$ (12) $\displaystyle\alpha_{2}=\delta-\gamma,\quad\beta_{2}=1-\delta$ $$

This gives

$$ $\displaystyle\boldsymbol{p}=(\gamma-\delta)\textbf{m}+\delta$ (13) $\displaystyle\boldsymbol{q}=(\delta-\gamma)\textbf{m}+1-\delta$ $$

### A.2 Hyperparameters

tablele 5 shows different hyperparameters that have been used for all the experiments using CLIP backbones. The hyperparameters were selected based on the performance of the first task of Stanford Cars dataset. All the results have been gathered over experiments averaged over 5 random seeds.

**Table 5: Hyper Parameters for all the experiments using CLIP ViT-B/16 model.**
| Hparams | CLIP model |
| --- | --- |
| Batch Size | 64 |
| Optimizer | AdamW |
| Learning Rate | $7.5e-6$ |
| CL Epochs | 10 |
| Buffer | 0 |
| TTL batch size | 64 |
| Momentum-EMA($\delta,\gamma,\lambda$) | 0.9999, 0.8, 0.9 |
| sparsity($c$) | 0.1 |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: 1. Are the comparisons with existing works fair?

Existing work such as L2P, DualPrompt, SLCA utilized a pretrained ViT model, i.e. ViT-B/16, which is trained on classification tasks, whereas the proposed method leveraged CLIP-ViT-B/16, which is trained on image and text pairs. Did the authors repeat experiments of existing methods with the same backbone? If the backbones are different, it is not convincing to conclude the superiority in performance in Table 2. In addition, the learning strategy of existing methods is also quite diverse: L2P and DualPrompt are prompt-based CL methods, whose backbone is frozen, and SLCA fine-tunes the backbone, ZSCL leverages external unlabeled datasets for finetuning. It seems unreasonable to put all these methods together and directly compare them with DoSAPP. 


2. Why is the ablation study of single momentum + mask union missing in Table 3?

Although it was illustrated in lines 462-464 in the text, the scores are not presented in the table.

3. How is the self-labeling (line 422) done?

It seems that the authors used the fine-tuned model (line Finetune in Tab 2) to do pseudo-labeling, is that correct? Even though this seems a valid choice, an intuitive adaptation of existing methods to the test-time adaptation scenario is to use existing methods, e.g. SPU or DualPrompt, and use their trained model after each task to perform pseudo labeling on test data and further adapt these models as test-time adaptation (TTA). That might be a more comparable baseline in this novel scenario.

4. Some small typos need to be corrected

line 215 'were' -> 'where'

table 2, Acc. of CIFAR100, ZSCL should be bold instead of DoSAPP, similarly, Acc of GTSRB should be L2P, F. of GTSRB should be SLCA.
soundness: 4
strengths: 1. The proposed test-time adaptation to mitigate forgetting is novel, interesting and realistic.

2. The paper is easy-to-follow, well presented

3. The proposed method is simple but effective
summary: This paper proposed to leverage the test-time data in CL to mitigate forgetting, which is novel, interesting and realistic. The authors further proposed a student-teacher learning framework to effectively learn and retain knowledge in this scenario. Specifically, the proposed DoSAPP(Double Smoothing via affine projected parameters) method consists of two stages: a supervised learning stage as in conventional CL and an unsupervised adaptation stage for test-time adaptation. In the former, the student would selectively update its parameters based on the gradient magnitude, and the teacher is a weighted exponential smoothing of the student model with dual momentum. In the latter, they first pseudo-label the test-time data based on the confidence of teacher and student models and then apply a similar learning paradigm as in the supervised learning stage. The authors validate the effectiveness of the proposed method in various datasets.
weaknesses: Some experimental comparisons and ablation studies are missing and some additional details need to be added. Please refer to the Questions section.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 1
questions: Please refer to the weakness section.
soundness: 3
strengths: 1. The paper is well-motivated. 
2. The idea of using test-time data to alleviate forgetting is interesting, and the performance is attractive. 
3. It is surprising to have a positive backward transfer, thanks to test time data.
summary: The paper proposes to use the test time data to alleviate the forgetting problem in the class incremental learning problem. The proposed method includes two training phases: a supervised learning phase using labeled training data, and an unsupervised learning phase using test time data. With the proposed DoSAPP, the forgetting problem can be greatly alleviated, and thus achieve a promising final performance.
weaknesses: **Major:**
1. It can be beneficial to have an ablation study about the size of test-time data $D^u$, with respect to the final performance.

2. It is suggested to include more information about the experiments. For example, the error bars of major experimental results in Table 2. Also, it is suggested to include detailed training time information (comparisons with other methods, training time for supervised and unsupervised phases)

3. The presentation of the paper is a concern. There are significant formatting issues/typos/non-completed equations/informal telegrapheses in the writing. A careful overhaul or English proofreading is suggested.

**Minor:**
1. It can be interesting to expand the experiments from pretrained CLIP to conventional classification backbones, like ImageNet-21k pretrained encoders and FC classifiers.

2. The equations should also be properly punctuated. 

**Summary:**

The paper presents an attractive setting and a novel view of alleviating forgetting in the CIL problem. However, there are significant issues, which limit the readability and the reproducibility, and thus I lean towards rejection. I'm happy to update my score if the concerns are properly addressed.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: I do not have questions for the authors.
soundness: 2
strengths: - The setting explored by the authors is noteworthy and realistic.
- The approach is technically solid.
- The concept of dual momentum is promising, though I have a few questions for the authors (see below).
summary: The authors propose a novel approach to Continual Learning that enables the learner to benefit from unsupervised data available during the test phase, addressing a challenge known as Test-Time Adaptation (TTA). The approach can be divided into two main contributions: the first focuses on mitigating catastrophic forgetting during training on sequential tasks, while the second introduces test-time adaptation (applied during the evaluation phase). Both processes leverage the well-established teacher-student paradigm, incorporating a dual mechanism to control the rate at which each weight of the teacher moves towards the corresponding weight of the student (dual momentum). Experiments demonstrate that the proposed architecture achieves good performance.
weaknesses: My primary concerns pertain to the **experimental setup**. Specifically, I found the comparison in Table 1 somewhat misleading. While DoSAPP leverages additional (albeit unsupervised) data from all tasks, the other approaches are standard continual learning methods that do not utilize test-time data. To their credit, the authors discuss this and include an additional comparison in Table 5 with a simple baseline combining SPU and pseudo-labeling. The authors justify this baseline by stating that SPU is the best-performing continual learning (CL) method; however, this claim does not hold up when examining the results in Table 2. For instance, SparseCL outperforms SPU, as does ZSCL (on average). Consequently, I feel the comparison is currently weak and should be expanded.
Additionally, while pseudo-labeling is a simple and effective approach, it only scratches the surface of Test-Time Adaptation (TTA), with many more advanced and effective methods proposed in recent years. Confidence in the technical contributions of this work (specifically, those outlined in Phase 2) would be strengthened if the authors extended their comparative analysis to include more recent TTA approaches. For example, the authors of CoTTA (Continual TTA) compared their approach not only with pseudo-labeling but also with BatchNorm adaptation and TENT (the latter outperforms pseudo-labeling).

I understand that these suggestions might sound like the typical “please, provide more experiments.” However, when a novel experimental setting bridges two or more fields, it is essential to design experiments that thoroughly consider the current state-of-the-art in each field.

Finally, I recommend that the authors correct the use of bold font in Table 2, where it should highlight top-performing methods rather than the proposed method. For instance, L2P achieves an accuracy of 75.45 on GTSRB, which is 3 points higher than DoSAPP (currently in bold). The same issue occurs with CIFAR-100 (where ZSCL performs better) and with the forgetting metric on Aircraft, CIFAR-100, CUB, and GTSRB.

**Clarity**
The concept of dual momentum is interesting, yet I believe there is a clarity issue in the paper. The authors state that a subset of parameters is frozen during training, following Zhang et al. (2023b). However, if certain student parameters remain unchanged during training, I would assume that their corresponding teacher parameters (updated via EMA) would also remain unchanged. This raises a question about how EMA could affect parameters that are kept frozen (line 311). I’m uncertain if this confusion arises from a lack of clarity in the paper or my own interpretation, but even after multiple readings, I could not resolve it.

**Presentation**

The paper is generally clear and well-written; however:
- In the section titled "Teacher Student Framework," the symbol indicating the parameter set is missing (see Eq. 3). Based on Fig. 2, it appears the symbol was \theta. I recommend the authors add it, as its absence complicates reading.
- Table 4 is presented inline within the text. This is acceptable, but it would be helpful if it appeared closer to its reference in Sec. 4.3.
- I suggest the authors refine punctuation and correct spacing between consecutive words, parentheses, citations, etc. For example, there are multiple instances where a blank space is missing between a word and a parenthesis.
- In Section A.3, the last line seems to be leftover text from a previous submission.

---

confidence: 4
contribution: 3
initial_rating: 3
presentation: 3
questions: - **Q1**: Is test time adaptation performed image-wise or batch-wise ?
- **Q2**: If it is performed batch wise, is it performed using the shuffled test set comprising all tasks at the same time, or is it performed task-by-task ?
soundness: 3
strengths: - Explores the intersection of test-time adaptation and continual learning which is an interesting direction
- Performs an ablation study
- Comparison with many existing methods
summary: The authors propose to leverage test time adaptation to improve the performance of foundational models when fine-tuned continually. They show significant improvements on many benchmarks when using their proposed TTA method.
weaknesses: - **W1**: More explanation around the exact setup used for test time adaptation needs to be made. For instance, it is not clear if Test time adaptation is done image-wise, in which case it would be fair to compare a test time adaptation method to another method that does not use test time adaptation, because both methods operate in the same setting. However, if this is not the case (for instance if test time adaptation is performed on batches that come from a single task, then evaluated on that task), this is not fair at all for other methods and any comparison with non-TTA method is doomed.
- **W2:** **Lack of baselines** Related to the previous weakness, in some TTA setting (that make sense in the wild), a lot of these CL methods you compare to could benefit from a simple change that would make them able to do TTA in some way (way less complicated than the method you propose). For instance, simply by allowing them to gather batch statistics in the BN layers if any, or by inferring the current task through retaining task prototypes and then only looking at the output of this tasks head.

I cannot review the paper further without having a clear answer and explanation inside the paper on how this TTA is performed, so I invite the authors to consider explaining this part very carefully. In particular, it should be made clear **in what order** the samples come to the model during the TTA phase, and when is the evaluation perfomed ? (after all samples have been processed or immediately after processing one sample)

Meta-review: The paper received quite diverse reviews. While reviewers 2tLK and 4MGu have given high scores, other two reviewers don't seem to prefer clear acceptance of this work. Upon reading myself the paper carefully and considering the points during discussions with other reviewers, unfortunately I'm recommending rejection of this work in its current form. Primary reasons for this rejection (most have already been discussed during the rebuttal) are mentioned below.

The proposed intuition and the method presented in the paper is **highly promising** and would strongly suggest the authors to incorporate the comments below to make it a solid submission in the future. 

Reasons for rejection:

---
- While the broad idea of utilizing test samples to mitigate forgetting seems promising, it is not entirely clear what led to the said design choices for TTA and why other CL methods do not benefit from it (reviewers uBk1 and PpH5 shared similar concerns). Though many questions relating this have been answered during the rebuttal, I (and other reviewers) however believe that the paper requires **serious rewriting** to ensure smooth and logical connection between thoughts. 

- Limited experiments. Current set of experiments are only performed using CLIP ViT-B/16 (similar to SPU). Since other CL baseline methods do not benefit from TTA (as mentioned by the authors during rebuttal), it becomes even more important that the proposed method is shown to perform well on a variety of settings to ensure its generality. For example: 
	- CLIP with RN50, ViT-B/32
	- FLAVA with ViT-B/16
	- vision only foundation models such as DINO and Masked Auto-Encoders

- Comparing the method with other baselines (involving large pre-trained models) would be valuable, for example RanPAC or even recently proposed continual fine-tuning methods.

- It would also be beneficial to discuss situations where both the student and the teacher are wrong during inference. Highly likely in open-world setting where test samples do not necessarily belong to the train distribution. In this case, other methods wouldn't suffer as they don't use test samples for adaptation, however, the proposed method might suffer. A discussion would be useful. 

[1] RanPAC: Random Projections and Pre-trained Models for Continual Learning

Decision: Reject