Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling

## Abstract

Abstract The majority of online continual learning (CL) advocates single-epoch training and imposes restrictions on the size of replay memory.
However, single-epoch training would incur a different amount of computations per CL algorithm, and the additional storage cost to store logit or model in addition to replay memory is largely ignored in calculating the storage budget.
Arguing different computational and storage budgets hinder fair comparison among CL algorithms in practice, we propose to use floating point operations (FLOPs) and total memory size in Byte as a metric for computational and memory budgets, respectively, to compare and develop CL algorithms in the same ‘total resource budget.’
To improve a CL method in a limited total budget, we propose adaptive layer freezing that does not update the layers for less informative batches to reduce computational costs with a negligible loss of accuracy.
In addition, we propose a memory retrieval method that allows the model to learn the same amount of knowledge as using random retrieval in fewer iterations.
Empirical validations on the CIFAR-10/100, CLEAR-10/100, and ImageNet-1K datasets demonstrate that the proposed approach outperforms the state-of-the-art methods within the same total budget. † † ∗ : Equal contribution. † † \leavevmode\nobreak\ \leavevmode\nobreak\ {}^{\dagger} start_FLOATSUPERSCRIPT † end_FLOATSUPERSCRIPT : Corresponding author.

## 1 Introduction

For the practicality of online continual learning (CL), the scenarios assumed by most online CL methods impose resource restrictions, such as the single training epoch and the size of the replay memory that limits the number of streamed samples that can be stored (Koh et al., ; Wang et al., ).
Although the ‘one-epoch training’ may give a rough sense of the computational constraint, the actual computation budget to train the models differs per method (Prabhu et al., ; Verwimp et al., ; Ghunaim et al., ) since each method requires a different amount of computations in a single epoch.
Several rehearsal-based CL methods require additional storage to store the previous models and logits (i.e., the unnormalized model output vector) (Buzzega et al., ; Koh et al., ; Zhou et al., ), which was usually not included in the memory budget, which mainly considers the size of episodic memory to store samples in previous tasks.
To this end, we compare CL methods with the same computational and memory budget considering all storage and computational costs.
We argue that the *total budget* of memory and computation will ensure the practicality of the proposed online CL algorithms.

For a fair comparison in computational budget, we use training FLOPs per sample as the metric instead of the number of epochs, since some methods require a lot of computations in a single epoch, while others do not.
FLOPs is an exact computational budget regardless of implementation details (Korthikanti et al., ), following (Zhao et al., ; Ghunaim et al., ).

For the same memory budget, we use an aggregated budget for various forms of extra storage, including replay memory, logits, and model parameters.
Following (Zhou et al., ), we convert all additional storage costs into byte and sum them up to obtain the actual memory cost.
Upon comparing the results under fair cost, we found that the performances were often different from what was reported in the original papers proposing each method.

Figure: Figure 1: Comparison of CL methods w/o total constraint (left) and w/ total constraint (right) on CIFAR-10 Gaussian setup. In the left plot, we compare CL methods with the same number of iterations and the same episodic memory size, i.e., conventional setup. In the right plot, we compare CL methods with the same training FLOPs per sample and a fixed storage budget that includes both episodic memory and model storage, i.e., our total-constrained CL setup. Compared to the conventional setup, aL-SAR shows improved performance under the total-constrained setup, since it can utilize the saved computational cost for additional model training.
Refer to caption: x1.png

Taking into account the total memory and computational budget, we additionally propose an online CL method using a computation-aware layer freezing strategy which only requires an additional cost of 0.02% compare to the replay memory size in ImageNet).
Specifically, we propose to *selectively* learn (or freeze) layers per a mini-batch based on the amount of information already learned in the previous mini batches, to reduce computations. In particular, as the number of layers being frozen increases, training computation can be saved at the cost of losing information for the mini-batch since frozen layers cannot acquire any information.
Considering the trade-off between computation and information, we propose ‘adaptive layer freezing’, which chooses the best layers to freeze by maximizing the Fisher Information (FI) gained by the model for each batch, given a fixed computation budget.
Unlike previous freezing methods (Lee et al., ; Hayes et al., ; Yuan et al., ), which predefine the rule of when and which layers to freeze, not generalizing to various datasets and models, we determine the appropriate layers to freeze in each forward pass considering the varying information contained in each batch.

For the loss of accuracy by the reduced computational cost, we propose a novel sample retrieval method to improve accuracy with negligible training cost.
While several online CL methods, such as MIR (Aljundi et al., ) and ASER (Shim et al., ), aim to retrieve more informative training batches than random retrieval from replay memory, these methods require forward/backward passes of a large candidate set to select informative batches.
This results in significantly higher computational overhead, with MIR and ASER requiring 4 times and 3 times more computation than ER, respectively.
The application of these methods would nullify the effort of reducing the computation by the proposed adaptive layer freezing method.
To this end, we propose to retrieve samples that the model has not learned much about information stored in episodic memory for training without incurring additional computational costs.
To quantify the degree of learning, we employ the frequency of recent usage of each sample in training and the similarity of the gradients between classes.
They are acquired during the training process, i.e., no need for additional inference.

For our empirical validations, we compare the state-of-the-art online CL algorithms under the same FLOPs of computations and the same bytes of storage in Fig. 1.
We observe that several high-performance CL methods do not maintain competitiveness under fixed FLOPs and memory budget, unexpectedly trailing behind a simple Experience Replay(Rolnick et al., ).
On the contrary, the proposed method outperforms them by a noticeable margin under the same total budget.

We summarize our contributions as follows:

- •
Proposing to measure computational and memory budgets of CL algorithms by using training FLOPs and total memory size in Bytes, to fairly compare different algorithms.
- •
Proposing a computationally efficient adaptive layer freezing that maximizes Fisher Information per computation, as well as a memory retrieval strategy that promotes the retrieval of samples that the model has not learned much.
- •
Empirical analysis on the computational and memory costs of various CL algorithms, showing that many state-of-the-art CL methods are less beneficial under the same budget and showing that the proposed method outperforms them by a noticeable margin across multiple benchmarks.

## 2 Related Work

### 2.1 Online Continual Learning with Memory Budget

Replay-based online CL methods use episodic memory and consider the memory budget.
Since we also consider using episodic memory, we review them in detail as follows.
Replay-based methods (Aljundi et al., ; Bang et al., ; Koh et al., ) store part of the past data stream in episodic memory to replay them in future learning.
Although there are simple sampling strategies such as random sampling (Guo et al., ) and reservoir sampling (Vitter, ), they are often insufficient to adapt to changing data distributions.
Rather than simple methods, researchers have developed advanced sampling strategies considering factors such as uncertainty (Bang et al., ), importance (Koh et al., ), and gradient (Tiwari et al., ).
However, these advanced methods often entail a high computational overhead, making them impractical for deployment in real-world applications.
RM (Bang et al., ) requires a significant amount of computational cost to calculate the uncertainty for diversified sampling.
Similarly, CLIB (Koh et al., ) involves additional forward and backward passes to calculate the decrease in loss for each batch iteration.

In addition to the memory management schemes, researchers investigate the memory usage schemes, i.e., sample retrieval strategies from the rehearsal buffers.
In addition to random retrieval (Chaudhry et al., ), the determination of retrieval based on the degree of interference (Aljundi et al., ) and the adversarial Shapley value (Shim et al., ) has been explored.
However, such methods require an inference of candidate samples, which leads to a nontrivial amount of computation in computing the loss (Aljundi et al., ) or the Shapely value (Shim et al., ).

### 2.2 Layer Freezing

Freezing layers have been investigated to reduce computational costs during training in joint training (i.e., ordinary training scenario other than CL) (Brock et al., ; Xiao et al., ; Goutam et al., ).
A common freezing approach (Wang et al., ; Li et al., ) includes determining whether to freeze a layer, based on the reference model and representation similarity, such as CKA (Cortes et al., ) and SP loss (Tung & Mori, ).
Additionally, EGERIA (Wang et al., ) unfreezes layers based on changes in the learning rate.

However, in CL, both online and offline, it is challenging to determine when to freeze a layer because metrics such as Euclidean distance and CKA cannot be used to compare the degree of convergence compared to the reference model (Mirzadeh et al., ).
Additionally, continual learning involves a non-*i.i.d*. setup, where the data distribution continues to change (Criado et al., ).
Therefore, in addition to changes in learning rate, it is important to consider the current data distribution when determining whether to freeze or unfreeze a layer in continual learning.
(Hayes et al., ) have explored freezing methods for continual learning. However, they use predefined freezing configurations such as the freezing backbone block 0 after task 1, while our freezing method adaptively freezes the layers using information per batch.

## 3 Approach

For efficient learning in computation and storage budget, we consider two strategies; (1) reducing the computational cost of each iteration and (2) reducing the number of iterations.
To implement both strategies, we propose a method employing two techniques; (1) adaptive layer-freezing and (2) similarity-aware retrieval of samples from episodic memory.

Figure: Figure 2: Gradient update procedure of the proposed aL-SAR. The colors in the ‘Similarity-Aware Retrieval’ box denote different classes. (1) ‘Retrieval Probability’ is calculated using class similarity $S$ and discounted use frequency $c_{i}$, where $c_{i}$ tracks the number of times the $i^{\text{th}}$ sample has been used for training. (2) A batch is sampled from memory by the ‘Retrieval Probability’ and (3) $c_{i}$ is updated by retrieval results. After the (4) forward pass of the model with the batch, (5) we compute the freezing criterion $(\text{BFC})_{n}$ for each layer $n$ of the model, using Fisher Information (FI) and $\frac{\partial\ell}{\partial x_{L}}$ (Sec. 3.1). (6) Layers 1 to $n_{\text{max}}=\operatorname*{arg\,max}_{n}(\text{BFC})_{n}$ (in this example, $n_{\text{max}}=2$) are frozen in the (7) backward pass. (8) $S_{ij}$ and FI are updated using the gradient $\frac{\partial\ell}{\partial\theta}$ obtained from the backward pass.
Refer to caption: x2.png

Specifically, for every training batch, the adaptive layer freezing method adaptively freezes layers so that the amount of information that can be gained from the mini-batch is maximized relative to the required computation.
The memory retrieval method retrieves training batches that the model has not learned sufficiently using the number of times each sample has been used for training, i.e., use-frequency, and class-wise gradient similarity.
This allows the model to learn the same amount of knowledge as using random retrieval in fewer iterations, consequently reducing the overall number of training iterations.
We call our method adaptive Layer freezing and Similarity-Aware Retrieval (aL-SAR), illustrating the gradient update procedure of the proposed aL-SAR in Fig. 2 and providing a pseudocode in Sec. A.3.

### 3.1 Adaptive Layer Freezing for Online CL

To reduce the computational cost of learning a neural network with minimal accuracy loss, there have been several studies on the freezing of neural network layers in non-CL literature (Liu et al., ; He et al., ; Wang et al., ).
These methods often rely on the learning convergence of each layer to determine which layers to freeze since converged layers no longer require further training.
However, in online CL, achieving convergence is often challenging due to the limited training budget and the ever-evolving training data distribution.
This requires a new approach to determine when and which layers to freeze for the incoming data in the online CL scenarios.

#### Selectively Freezing Layers by Maximum Fisher Information (FI).

For a computationally efficient freezing criterion in online CL, we propose to freeze layers that learn little information per computational cost by measuring ‘information’ ($I$) gained by each layer during training.
Here, we define the information ($I$) using Fisher Information (FI), since it is a widely used metric to measure the amount of information that each parameter in a neural network obtains from data (Durant et al., ; Desjardins et al., ; Ollivier, ).
So, we use FI to measure the layer-wise information that each layer gains from the input data to determine which layers to freeze.
Note that freezing some layers facilitates training a model for more mini-batches within a fixed computational budget as it reduces the computations per mini-batch.

However, as a trade-off, as the number of frozen layers increases, the number of updated parameters decreases, reducing the amount of information obtained per mini-batch.
To maximize the information ($I$) in the model while minimizing the computational cost ($C$), we propose to maximize the expected amount of information gained per computation $(I/C)$.

Formally, we try to find $n_{\text{max}}=\operatorname*{arg\,max}_{n}(I/C)_{n}$ for $n\in[1,L]$ where we define $(I/C)_{n}$ as the amount of information gained per computational cost when updating the model with layers 1 to $n$ frozen. $L$ refers to the total number of layers.
To compute $(I/C)_{n}$, we factorize it as:

$$ $(I/C)_{n}=(I/\text{mb})_{n}\cdot(\text{mb}/C)_{n},$ (1) $$

where ‘mb’ refers to the mini-batch.
$(I/\text{mb})_{n}$ and $(\text{mb}/C)_{n}$ represent the amount of information gained per mini-batch and the number of mini-batch iterations per computation, respectively, when layers 1 to $n$ are frozen.

#### Amount of Information Gained per Mini-batch ( I / mb ) n subscript 𝐼 mb 𝑛 (I/\text{mb})_{n} ( italic_I / mb ) start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT .

To compute $(I/\text{mb})_{n}$, we use $F(\theta_{i})$, Fisher Information Matrix $F(\theta)$ of layer $i$, where $\theta$ and $\theta_{i}$ denote the parameters of the model $p_{\theta}(\cdot)$ and the parameters of the layer $i$ of $p_{\theta}(\cdot)$, respectively.
But computing all components of $F(\theta_{i})$ is costly as Hessian is required, which involves second-order derivatives and can be computationally inefficient.
To avoid the cost, we use first-order approximation of $F(\theta_{i})$ by using the diagonal components of the $F(\theta_{i})$ following (Kirkpatrick et al., ; Soen & Sun, ), i.e., using the trace operator $\text{tr}(\cdot)$ as:

$$ $(I/\text{mb})_{n}=\sum_{i=n+1}^{L}\text{tr}(F(\theta_{i})),\leavevmode\nobreak \ \leavevmode\nobreak\ \text{where}\leavevmode\nobreak\ \leavevmode\nobreak\ F (\theta_{i})=\mathbb{E}_{p_{\theta}(z),\,z\in\mathcal{D}}\left[\left(\frac{ \partial\ell}{\partial\theta_{i}}\right)\cdot\left(\frac{\partial\ell}{ \partial\theta_{i}}\right)^{\intercal}\right],$ (2) $$

where $\mathcal{D}$ is the data stream, $z\in\mathcal{D}$ is the training data batch in the data stream, and $\ell=\log p_{\theta}(z)$ is the loss function.
Note that since layers 1 to $n$ are frozen, the model gains information of layers $n+1$ to $L$, the unfrozen layers only.

#### Number of Mini-batch per Computational Cost ( mb / C ) n subscript mb 𝐶 𝑛 (\text{mb}/C)_{n} ( mb / italic_C ) start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT .

To compute $(\text{mb}/C)_{n}$, we initially determine its inverse, denoted $(C/\text{mb})_{n}$, which represents the computational cost per mini-batch when layers 1 to $n$ are frozen.
We calculate $(C/\text{mb})_{n}$ by splitting it into forward and backward propagation as: $(C/\text{mb})_{n}=\sum_{i=1}^{L}\text{(FF)}_{i}+\sum_{i=n+1}^{L}\text{(BF)}_{i}$,
where $\text{(FF)}_{i}$ and $\text{(BF)}_{i}$ refer to the forward FLOPs and the backward FLOPs of layer $i$, respectively.
By taking an inverse of it, we obtain

$$ $(\text{mb}/C)_{n}=\frac{1}{\sum_{i=1}^{L}\text{(FF)}_{i}+\sum_{i=n+1}^{L}\text {(BF)}_{i}}.$ (3) $$

Note that since layers 1 to $n$ are frozen, no backward computation is performed for those layers.
When the number of frozen layers (i.e., $n$) increases, the number of layers performing backward operations is reduced, i.e., $\sum_{i=n+1}^{L}\text{(BF)}_{i}$ decreases, thus leading to an increase in the possible number of mini-batch within the given computational budget $C$.

Combining Equation 2 and Equation 3, we can finally calculate $(I/C)_{n}$, which represents the information the model can gain within a given computational budget when layers 1 to $n$ are frozen,
by a product of $(I/\text{mb})_{n}$ and $(\text{mb}/C)_{n}$:

$$ $(I/C)_{n}=(I/\text{mb})_{n}\cdot(\text{mb}/C)_{n}=\frac{\sum_{i=n+1}^{L}\text{ tr}(F(\theta_{i}))}{\sum_{i=1}^{L}\text{(FF)}_{i}+\sum_{i=n+1}^{L}\text{(BF)}_ {i}}.$ (4) $$

By freezing layer 1 to layer $n_{\text{max}}=\operatorname*{arg\,max}_{n}(I/C)_{n}$, we can maximize the expected amount of information gained per computation during training.

#### Batch-wise Freezing for Online CL.

$(I/C)_{n}$ is supposed to be calculated on the entire data stream since $F(\theta)$ is defined as an expectation over the whole dataset in Equation 2.
Thus, $(I/C)_{n}$ does not account for the variable amount of information of each batch.
In online CL where the incoming batch distribution continuously shifts, the variation is not negligible.
Specifically, for batches containing sufficiently trained classes, it is advantageous to freeze more layers, while for batches with insufficiently trained classes, it is beneficial to freeze fewer layers.
To address this, we propose the batch freezing criterion (BFC) which quantifies the *net benefit* in the information gained by the model when we freeze layers given an input batch $z_{t}$.

To define the BFC, we compare (1) the amount of information we would lose from the current batch by freezing and (2) the expected amount of information we would gain in the future using the saved computation from freezing.
Then, we can estimate the net benefit from freezing in terms of the information gained by the model by subtracting (1) from (2).

To estimate (1), we use the trace of layer-wise FI, i.e., $\text{tr}(F(\theta_{i}))$, defined in Equation 2.
Note that $F(\theta_{i})$ is the FI over the whole dataset; thus we convert it into batch-specific FI $F_{z_{t}}(\theta_{i})=\mathbb{E}_{p_{\theta}(z),\,z\in z_{t}}\left[\left(\frac
{\partial\ell}{\partial\theta_{i}}\right)\cdot\left(\frac{\partial\ell}{
\partial\theta_{i}}\right)^{\intercal}\right]$ by multiplying the ratio of the FI of the batch relative to the FI of the entire dataset.
Using $F_{z_{t}}(\theta_{i})$, the amount of information we would lose from the current batch $z_{t}$ by freezing layers 1 to $n$ is obtained as $\sum_{i=1}^{n}\text{tr}(F_{z_{t}}(\theta_{i}))$.
Please refer to Sec. A.1 for more details on the estimation of $F_{z_{t}}(\theta_{i})$ from $F(\theta_{i})$.

To estimate (2), we calculate the amount of computations saved by freezing layers 1 to $n$, and the amount of information we can obtain from the saved computations.
Here, the saved computation refers to the sum of the backward FLOPs of the frozen layers, denoted as $\sum_{i=1}^{n}(\text{BF})_{i}$.
Using $(I/C)_{n}$ defined in Equation 4, we estimate the expected information ($I$) obtainable from the saved computations ($C$) by multiplying the saved computation and $(I/C)_{n}$.
With optimal freezing that maximizes $(I/C)$, the anticipated information obtained from the saved calculations is $\max_{m}{(I/C)_{m}}\cdot\sum_{i=1}^{n}\text{(BF)}_{i}$.

Finally, by subtracting (1) from (2), we obtain $\text{BFC}(z_{t})_{n}$ as:

$$ $\displaystyle\text{BFC}(z_{t})_{n}=$ $\displaystyle\max_{m}{(I/C)_{m}}\cdot\sum_{i=1}^{n}\text{(BF)}_{i}-\sum_{i=1}^ {n}\text{tr}\left(F_{z_{t}}(\theta_{i})\right).$ (5) $$

The positive $\text{BFC}(z_{t})_{n}$ implies that freezing layer 1 to $n$ is beneficial in terms of information, and a negative value indicates otherwise.
By freezing layer 1 to $n_{\text{max}}(z_{t})=\operatorname*{arg\,max}_{n}\text{BFC}(z_{t})_{n}$, we can select the most beneficial freezing strategy for batch $z_{t}$.
We argue that it allows us to dynamically select the freezing strategy by the learning capacity of the layers and the batch’s informativeness.
We empirically demonstrate the distribution of the $\text{BFC}(z_{t})_{n}$ in Sec. A.10.

### 3.2 Similarity-Aware Retrieval based on ‘Use-Frequency’

In rehearsal-based CL methods, sample retrieval strategies such as MIR (Aljundi et al., ) and ASER (Shim et al., ) have a high computational cost, requiring multiple additional model inferences.
Thus, in the same computational budget, their performances are surprisingly inferior to simple random retrieval (i.e., ER) as shown in Fig. 1.
Here, we propose a computationally efficient sample retrieval strategy that does not require additional model inference.

In online CL, new data continuously stream in, and old data remain in memory, causing an imbalance in ‘the number of times each sample is used for training’, which we call ‘*use-frequency*.’
We argue that samples with high use-frequency yield marginal knowledge gains in training, and samples with low use-frequency are likely to contain knowledge that the model has not yet learned.
So, we propose to sample data with low use-frequency for training with high probability.

#### Discounted Use Frequency ( c i subscript 𝑐 𝑖 c_{i} italic_c start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ).

But using the use-frequency for sampling does not consider the knowledge forgetting in the CL setup.
If a sample was frequently used in the past but seldom used in recent iterations, its knowledge may have been forgotten, despite its high use-frequency.
Inspired by the exponential decaying model of forgetting  (Shin & Lee, ; Mahto et al., ; Chien et al., ), we propose a decay factor ($0<r<1$) in the use frequency at each iteration, calling *’discounted-use-frequency’* for $i^{\text{th}}$ sample ($c_{i}$).
For example, if $i^{\text{th}}$ sample is used $n$ times for training at a specific time point, after $t$ iterations, we define its discounted use frequency as $c_{i}=n\cdot r^{t}$.

#### Effective Use Frequency ( c ^ i subscript ^ 𝑐 𝑖 \hat{c}_{i} over^ start_ARG italic_c end_ARG start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ).

However, the model can learn knowledge about a sample by training other similar samples (e.g., samples from the same classes).
Thus, $c$ of the other samples could also affects $c_{i}$ of the sample.
These similar samples *effectively* increase the use-frequency of the particular sample.
At the same time, the model may lose knowledge about the sample when training on different samples (e.g., from other classes), effectively decreasing the use-frequency.

To account for this, we define *‘effective-use-frequency’* by adding the other samples’ use-frequency multiplied by the similarity of samples to $c_{i}$.
For the sample similarity score, inspired by (Du et al., ), which uses gradient similarity to assess the helpfulness or harmfulness of an auxiliary task to the original task, we hypothesize that samples with similar gradients bear similar information.
So, for the proxy of sample similarity, we use cosine similarity between the gradients.

However, tracking the gradient similarities between all sample pairs requires excessive memory ($\sim 10^{12}$ pairs for ImageNet) and computation.
Thus, we approximate it to class-wise similarities, which is the expected gradient similarity between samples from two classes.
Formally, we define the class-wise similarity $\mathcal{S}_{y_{1},y_{2}}$ for classes $y_{1}$ and $y_{2}$ as:

$$ $\mathcal{S}_{y_{1},y_{2}}=\mathbb{E}_{z_{1}\in D_{y_{1}},z_{2}\in D_{y_{2}}} \left[\cos(\nabla_{\theta}l(z_{1}),\nabla_{\theta}l(z_{2}))\right],$ (6) $$

where $D_{y_{i}}$ is the training data for class $y_{i}$ and $\nabla_{\theta}l(z_{i})$ is gradient of $i^{\text{th}}$ sample.
Using the approximated class-wise similarities, we define the effective-use-frequency $\hat{c}_{i}$ for the $i^{\text{th}}$ sample as:

$$ $\hat{c}_{i}=c_{i}+\sum_{y\in\mathcal{Y}}\mathcal{S}_{y,y_{i}}\cdot C_{y},$ (7) $$

where $\mathcal{Y}$ is the set of all seen classes, $\mathcal{S}_{y,y_{i}}$ is a class similarity between class $y$ and $y_{i}$, and $C_{y}=\sum_{y_{j}=y}c_{j}$ is the sum of the discounted use-frequencies for all samples of class $y$.

Unfortunately, calculating the expected value in $\mathcal{S}_{y_{i},y_{j}}$ (Equation 6) from scratch for each iteration requires a gradient calculation for all samples in the classes $y_{i}$ and $y_{j}$, which is computationally expensive.
As a computationally efficient alternative, we further propose using the EMA to update the previous estimate of ${S}_{y_{i},y_{j}}$ rather than calculating the expectation from scratch.
Note that we only utilize gradients from unfrozen layers, obtained during the training process, i.e., incurring no additional cost.
Please refer to Sec. A.17 for detailed information on the calculation of $\mathcal{S}_{y_{i},y_{j}}$.

Finally, we obtain the retrieval probabilities $p_{i}$ for $i^{\text{th}}$ sample with the effective-use-frequency as:

$$ $p_{i}=\frac{e^{-\hat{c}_{i}/T}}{\sum_{j=1}^{|\mathcal{M}|}e^{-\hat{c}_{j}/T}},$ (8) $$

where $T$ is a temperature hyperparameter.
Samples with low $\hat{c}_{i}$ have high chances of being retrieved, so insufficiently trained samples are preferred to sufficiently trained ones, thereby accelerating training.
We present several toy experiments and an ablation study on the components of SAR to validate the empirical benefit of our retrieval algorithm in Sec. A.24 and Sec. A.8, respectively.

## 4 Experiments

### 4.1 Setup

For empirical validation, we adopt the total budget for memory and computation.
For the memory budget, we use Bytes following (Zhou et al., ), which considers memory costs not only for the samples in episodic memory but also for additional model parameters used in regularization or distillation.
For the computational budget, we use training FLOPs.
For the dataset, we use CIFAR-10/100, CLEAR-10/100, and ImageNet-1K.
We evaluate the methods in a conventional disjoint task setup and a newly proposed Gaussian task setup (Shanahan et al., ; Wang et al., ; Koh et al., ).
For all experiments, we averaged 3 different random seeds, except ImageNet-1K due to computational cost (Bang et al., ; Koh et al., ).
We conducted a Welch’s $t$-test with a significance level of 0.05.
We highlighted the highest performance in bold. In cases where statistical significance was not observed, we underlined all other results within the significance level.

#### Metrics.

We report the last accuracy $A_{\text{last}}$ and the area under the curve of accuracy $A_{\text{AUC}}$ (Koh et al., ).
The $A_{\text{last}}$ measures the accuracy at the end of CL.
The $A_{\text{AUC}}$ measures the accuracy per time step using the accumulated test set of all previously seen tasks and then computes the area under the accuracy curve.
For each evaluation, we evaluate using the entire test set for the classes seen so far.
We argue that $A_{\text{AUC}}$ is a suitable metric to measure prompt learning of new knowledge.

#### Baselines.

We compare aL-SAR to state-of-the-art online CL methods such as ER (Rolnick et al., ), DER++ (Buzzega et al., ), MIR (Aljundi et al., ), MEMO  (Zhou et al., ), REMIND (Hayes et al., ), EWC (Kirkpatrick et al., ), OCS (Yoon et al., ), LiDER (Bonicelli et al., ), X-DER (Boschini et al., ), CCL-DC (Wang et al., ), and CAMA (Kim et al., ).
We describe the implementation details and hyperparameters in Sec. A.4.

### 4.2 Quantitative Analysis

We evaluated CL methods, including aL-SAR, with strictly restricted computation and memory budgets as specified in Sec. 4.1.
Note that while we set the training iterations of aL-SAR to be the same as other baselines, aL-SAR adaptively freezes layers, resulting in fewer FLOPs consumed.
For experiments on various computational and memory budgets, we use the relatively small CIFAR datasets to cover a wide range of given budgets.
To validate our methods on large datasets and datasets with temporal domain shift, we also show experiments on ImageNet and CLEAR datasets.
Note that these experiments are conducted on various CL setups, including Gaussian task setup, Disjoint task setup, and domain-incremental setup.

#### Various Computational Budget under the Same Memory Budget.

We compare CL methods under fixed memory budgets and various computational budgets in Fig. 3.
We observe that aL-SAR significantly outperforms others in all datasets and both setups, especially under a low computational budget.
It shows that our similarity-aware retrieval effectively promotes rapid learning by retrieving informative training batches even with a limited computational budget, while random retrieval and MIR (Aljundi et al., ) require high computation to achieve comparable performance.

Figure: Figure 3: Accuracy on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 for a wide range of FLOPs per sample. aL-SAR outperforms all CL methods compared. We use a memory budget of 30.4MB.
Refer to caption: x3.png

Furthermore, we observe a notable increase in the FLOPs saved by aL-SAR through freezing, particularly pronounced at higher computational budgets.
As the model undergoes more iterations, the amount of information that the model gains from the training data decreases.
Thus, our adaptive layer freezing adaptively adjusts the freezing criterion to freeze more layers, leading to lower FLOPs, thus the line stops at the earlier GFLOPs value than the baselines.
We provide comprehensive analysis when varying computations under various memory constraints in Sec. A.5.
Please refer to Sec. A.18 for more details on the computational budget.

Figure: Figure 4: Accuracy on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 for various memory budget. aL-SAR outperforms all CL methods compared. We use a computational budget of 171.94 TFLOPs.
Refer to caption: x5.png

#### Various Memory Budget under the Same Computational Budget.

We now fix the computational budget and test various memory budgets for CL methods in Fig. 4.
With its minimal additional memory usage and effective utilization of episodic memory through similarity-aware retrieval,
aL-SAR again outperforms other methods by a significant margin in all datasets, indicating its suitability for both large and small memory budgets.

#### Large Datasets and Temporal Domain Shifts.

To investigate the scalability of CL methods on large datasets and temporal domain shifts, we report the performances on ImageNet-1K and CLEAR-10/100 with a fixed computational and memory budget in Tab. 1.
In the ImageNet-1K Gaussian setup, aL-SAR outperforms other methods with slightly fewer FLOPs, and in the ImageNet-1K Disjoint setup, aL-SAR shows similar performance with other methods while using significantly fewer FLOPs.
In this setup, training distribution changes continuously, so the model has to constantly adapt to the new distribution, resulting in less freezing and more benefit from the fast adaptation enabled by the retrieval method.
In contrast, in the ImageNet-1K Disjoint setup, the distribution shift only occurs at the task boundaries, so fast adaptation by retrieval has a smaller effect, but aL-SAR freezes more layers after each task has been sufficiently learned.

**Table 1: Quantitative comparison between different CL methods on CIL setup. aL-SAR outperforms baselines with a lower computational budget and the same storage budget. Specifically, while all other baselines consume 112,000, 716, and 5,250 TFLOPs, aL-SAR consumes 93,000, 598, and 4,059 TFLOPs in the ImageNet-1K, CLEAR-10, and CLEAR-100, respectively. The memory budget is fixed at 5,736MB, 564MB, and 1,148MB in ImageNet-1K, CLEAR-10, and CLEAR-100, respectively.**
| Methods | ImageNet-1K | CLEAR-10 | CLEAR-100 |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Gaussian Setup | Disjoint Setup |  |  |  |  |  |  |  |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ |  |
| EWC (Kirkpatrick et al., ) | $20.05$ | $32.55$ | $44.17$ | $36.11$ | 62.40$\pm$0.17 | 71.25$\pm$1.29 | 29.62$\pm$0.44 | 40.36$\pm$1.34 |
| ER (Rolnick et al., ) | $20.01$ | $32.60$ | $44.13$ | $36.85$ | 64.00$\pm$0.06 | 72.28$\pm$1.15 | 31.56$\pm$0.42 | 44.90$\pm$0.74 |
| ER-MIR (Aljundi et al., ) | $7.45$ | $17.04$ | $35.48$ | $35.83$ | 62.29$\pm$0.17 | 70.52$\pm$1.13 | 23.24$\pm$0.14 | 34.92$\pm$1.16 |
| REMIND (Hayes et al., ) | $15.58$ | $22.42$ | $37.42$ | $33.14$ | 62.64$\pm$0.04 | 70.68$\pm$0.90 | 26.76$\pm$0.30 | 30.99$\pm$0.55 |
| DER++ (Buzzega et al., ) | $23.51$ | $29.82$ | $40.30$ | $37.09$ | 61.30$\pm$0.77 | 71.52$\pm$1.02 | 25.83$\pm$0.43 | 39.89$\pm$0.89 |
| OCS (Yoon et al., ) | $6.90$ | $19.57$ | $27.50$ | $30.41$ | 58.69$\pm$0.25 | 68.69$\pm$1.21 | 15.24$\pm$1.27 | 22.52$\pm$2.02 |
| LiDER (Bonicelli et al., ) | $27.65$ | $34.66$ | $32.68$ | $31.73$ | 58.36$\pm$0.75 | 69.16$\pm$1.21 | 28.75$\pm$0.28 | 39.92$\pm$1.29 |
| X-DER (Boschini et al., ) | $27.46$ | $34.47$ | $32.34$ | $31.41$ | 58.19$\pm$0.23 | 69.70$\pm$0.97 | 23.01$\pm$0.15 | 33.70$\pm$0.70 |
| MEMO (Zhou et al., ) | $11.15$ | $19.98$ | $39.30$ | $33.35$ | 60.29$\pm$0.31 | 68.69$\pm$1.21 | 20.33$\pm$0.28 | 25.53$\pm$0.92 |
| CAMA (Kim et al., ) | $7.94$ | $17.44$ | $35.38$ | $36.02$ | 60.97$\pm$0.52 | 71.66$\pm$1.18 | 26.27$\pm$0.37 | 39.42$\pm$0.67 |
| CCL-DC (Wang et al., ) | $19.70$ | $32.12$ | $44.07$ | $35.98$ | 61.08$\pm$0.55 | 71.25$\pm$0.82 | 23.81$\pm$0.32 | 35.07$\pm$0.67 |
| aL-SAR (Ours) | 33.94 | 38.02 | 45.08 | 37.25 | 67.61$\pm$0.45 | 74.36$\pm$0.80 | 38.65$\pm$0.30 | 47.56$\pm$0.37 |

We then investigate CL methods under temporal domain shift with fixed computational and memory budget using the CLEAR-10/100 datasets.
Unlike the class-incremental, where new classes are added, this domain-incremental setup introduces new domains, while the classes remain the same.
As shown in the table, aL-SAR also outperforms the state-of-the-art in domain-incremental setups.
It is partly because our retrieval method balances the use-frequency of samples in different domains so that the model learns more on relatively less-learned domains, allowing fast adaptation to new domains.
Note that aL-SAR also significantly saves FLOPs thanks to adaptive layer freezing.
Please refer to Sec. A.19 and Sec. A.18 for more details about the memory budget and the computational budget, respectively.
Moreover, we provide extensive experimental results in the memory infinite setup and comparisons with efficient CL baselines in Sec.A.20 and Sec.A.23, respectively.

#### Comparison of aL with Layer Freezing Methods.

We compare our proposed adaptive freezing method (aL) with REMIND (Hayes et al., ) and PTLF (Yang et al., ), layer freezing methods for offline CL, as well as EGERIA (Wang et al., ), a layer freezing method for non-CL.
The results are summarized in Tab. 2.
As we can see in the table, PTLF and EGERIA show negligible differences in TFLOPs with No Freezing, indicating minimal layer freezing.
While REMIND significantly reduces training FLOPs, it leads to performance degradation.
In contrast, our proposed aL significantly reduces training FLOPs while maintaining performance.
Notably, REMIND, PTLF, and EGERIA require task identity information during training, while aL operates without it. This further highlights the practicality and adaptability of aL in online CL setups.
We provide a detailed analysis of the results and further comparisons in the disjoint setup in Sec.A.13 for the sake of space.

**Table 2: Comparison between our proposed adaptive layer freezing and other freezing methods. We compare them in the CIFAR-10 Gaussian setup and the CIFAR-100 Gaussian setup.**
| Methods | CIFAR-10 | CIFAR-100 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| No freezing | $\textbf{64.60}\pm\textbf{0.83}$ | $72.43\pm 0.38$ | 171.94 | $\textbf{42.49}\pm\textbf{0.75}$ | $50.49\pm 0.29$ | 515.82 |
| REMIND | $59.25\pm 0.37$ | $65.01\pm 1.31$ | 151.31 (-12.0%) | $35.50\pm 0.56$ | $40.00\pm 0.86$ | 453.92(-12.0%) |
| PTLF | $63.82\pm 0.24$ | $71.30\pm 0.91$ | 164.14 (-4.5%) | $42.09\pm 0.58$ | $49.92\pm 0.65$ | 495.67 (-4.1%) |
| EGERIA | $63.95\pm 0.72$ | $71.72\pm 0.81$ | 169.25 (-1.6%) | $42.18\pm 0.63$ | $50.05\pm 0.72$ | 506.54 (-1.8%) |
| aL (Ours) | $64.38\pm 0.32$ | $\textbf{72.57}\pm\textbf{0.79}$ | 146.80 (-14.6%) | $42.38\pm 0.76$ | $\textbf{50.62}\pm\textbf{0.87}$ | 427.90 (-17.0%) |

#### Comparison of SAR with Memory Retrieval Methods.

We compare our proposed retrieval method, i.e., SAR with ASER (Shim et al., ) and MIR (Aljundi et al., ) in Sec. A.21.

#### Application of aL in Training Multi-Modal Large Language Models (MLLMs)

We demonstrate the effectiveness of aL in training MLLMs by applying it to training of the LLaVA-1.5-7B model (Liu et al., ).
Following Ye et al. (), we update only the pretrained projection MLP layers and LoRA adapters (Hu et al., ), keeping the LLM frozen for training efficiency.
We summarize the result in Tab. 3.
We provide implementation details and datasets (i.e., Bongard-HOI (Jiang et al., ) and Bongard-OpenWorld (Wu et al., )) of MLLM training in Sec. A.12.
As shown in the table, aL saves approximately 12% in training FLOPs while maintaining performance, by adaptively freezing LoRA layers during the training process.
We believe that aL can be effectively integrated in a plug-and-play manner with CL methods in large models, such as LLMs and MLLMs, to reduce training costs while achieving strong performance.

**Table 3: The effect of adaptive freezing on MLLM training. We use LLaVA-1.5-7B model with LoRA.**
| Methods | Bongard-HOI | Bongard-OpenWorld |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| No freezing | 60.31$\pm$1.28 | 49.63$\pm$1.71 | 1578.02 | 56.33$\pm$1.45 | 58.82$\pm$2.60 | 2959.61 |
| aL (Ours) | 60.52$\pm$1.50 | 50.83$\pm$3.27 | 1391.81 (-11.8%) | 57.69$\pm$3.01 | 57.79$\pm$3.55 | 2579.83 (-12.8%) |

### 4.3 Ablation Study

We ablate the model to investigate the benefit of each proposed component in CIFAR-10/100 and summarize the results in Tab. 4.
In Tab. 4, similarity-aware retrieval (SAR) increases the performance while using the same number of iterations.
This shows that SAR increases the amount of knowledge learned per iteration, as we claim in Sec. 3.
While computational cost also increases, its increase is modest compared to other retrieval methods such as MIR (Aljundi et al., ) or ASER (Shim et al., ) that require $3\sim 4\times$ more computations.
Furthermore, we observe that the aL significantly reduces FLOPs with negligible loss in accuracy.
In summary, our method outperforms the baseline while using fewer FLOPs than the baseline, each by a noticeable margin.
We provide further ablation studies of the proposed components in the Disjoint setup in Sec. A.7.
Furthermore, we present an ablation study of the proposed retrieval method, i.e., SAR, in Sec.A.8.

**Table 4: Benefits of the proposed components of aL-SAR, adaptive layer freezing (aL) and similarity-aware retrieval (SAR), in Gaussian task setup. ‘Vanilla’ is a simple replay-based method that trains on randomly retrieved batches from a balanced reservoir memory. The memory budget is 7.6MB for CIFAR-10 and 13.44MB for CIFAR-100. We train for 1 iter per sample for CIFAR-10 and 1.5 iter per sample for CIFAR-100.**
| Methods | CIFAR-10 | CIFAR-100 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| Vanilla | 60.76$\pm$0.11 | 70.08$\pm$0.97 | 163.74 | 31.97$\pm$0.89 | 37.80$\pm$1.30 | 245.91 |
| + aL | 60.38$\pm$0.54 | 69.04$\pm$0.83 | 142.23 | 31.77$\pm$0.60 | 38.03$\pm$0.35 | 217.40 |
| + SAR | 64.60$\pm$0.83 | 72.43$\pm$0.38 | 171.94 | 37.60$\pm$0.40 | 42.69$\pm$0.18 | 257.97 |
| + aL & SAR (Ours) | 64.38$\pm$0.32 | 72.57$\pm$0.79 | 146.80 | 37.20$\pm$0.73 | 42.55$\pm$0.79 | 221.49 |

Moreover, we provide detailed studies in the appendix for the space sake.
Specifically, we investigate the performance with different freezing strategies in Sec. A.6, the detailed effect of freezing on accuracy and FLOPs in Sec. A.9, the effect of temperature $T$ in Sec. A.22, and the application of our freezing method on Vision Transformer (ViT) (Dosovitskiy et al., )) is discussed in Sec. A.14.

## 5 Conclusion

We address the challenge of achieving high performance on both old and new data with minimal computational cost and limited storage budget in online CL.
While CL with fixed episodic memory size has been extensively studied, we have investigated the total storage budget required for online CL as well as the computational budget for developing practically useful online CL methods.

To this end, we proposed aL-SAR, a computationally efficient CL method comprising two components: similarity-aware retrieval and adaptive layer freezing.
Our empirical validations show that several high-performing CL methods are not competitive under a fixed computational budget, falling behind a simple baseline of training on randomly retrieved batches from memory.

## Ethics Statement

We propose a better learning scheme for online continual learning for realistic learning scenarios.
While the authors do not explicitly aim for this, the increasing adoption of deep learning models in real-world contexts with streaming data could potentially raise concerns such as inadvertently introducing biases or discrimination.
We note that we are committed to implementing all feasible precautions to avert such consequences, as they are unequivocally contrary to our intentions.

## Reproducibility statement

We take reproducibility in deep learning very seriously and highlight some of the contents of the manuscript that might help to reproduce our work.
We will definitely release our implementation of the proposed method in Sec. 3, the data splits and the baselines used in our experiments in Sec. 4.

## Appendix A Appendix

### A.1 Details on the estimation of batch-wise Fisher Information F z t ⁢ ( θ i ) subscript 𝐹 subscript 𝑧 𝑡 subscript 𝜃 𝑖 F_{z_{t}}(\theta_{i}) italic_F start_POSTSUBSCRIPT italic_z start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT end_POSTSUBSCRIPT ( italic_θ start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT )

To estimate the batch-wise FI $F_{z_{t}}(\theta_{i})$ from FI $F(\theta_{i})$, we compute the FI ratio, i.e., the ratio of the FI of the batch $z_{t}$ to the FI of the entire dataset, using the property that the FI is quadratically proportional to the magnitude of the gradient (equation 2).
To be specific, we compute the ratio between $|\nabla_{x}\ell(z_{t})|^{2}$ of the current batch and the expectation of $|\nabla_{x}\ell(z)|^{2}$ on the whole dataset.

Note that, we only use the gradient of the last layer feature $x_{L}$ to estimate the magnitude of the gradient following (Koh et al., ), since the gradients of the preceding layers are proportional to the gradient of the final layer due to the chain rule.
Using the estimate, we compute the batch-wise FI $F_{z_{t}}(\theta_{i})$ from batch $z_{t}$ when freezing layers 1 to $n$ as:

$$ $F_{z_{t}}(\theta_{i})=\frac{|\nabla_{x_{L}}\ell(z_{t})|^{2}}{\mathbb{E}_{z} \left[|\nabla_{x_{L}}\ell(z)|^{2}\right]}\cdot F(\theta_{i}),$ (9) $$

where $\mathbb{E}_{z}$ is the expectation over all input batches.

For the calculation of $F_{z_{t}}(\theta_{i})$, we have to compute the expected values in the average gradient magnitude ($\mathbb{E}_{z}\left[|\nabla_{x_{L}}\ell(z)|^{2}\right]$) and FI ($F(\theta_{i})$’s) (equation 2).
Since calculating the expected values (using all samples in replay memory) in every learning iteration is computationally expensive, we estimate them by the exponential moving average (EMA) of the estimated expectations computed by the mini-batch of the past iterations.
However, the EMA estimate of $\text{tr}(F(\theta_{i}))$ requires a gradient calculation for all layers, so it cannot be used with freezing, which stops the gradient computations.
Since the estimation of $\text{tr}(F(\theta_{i}))$ and the freezing cannot be performed at the same time, at each $m$ iteration, we train (i.e., unfreeze) all layers to update the estimate of $\text{tr}(F(\theta_{i}))$ for all $i$.
For the other $m-1$ iterations, we do not update $\text{tr}(F(\theta_{i}))$ and freeze the model based on the values of $(\text{BFC})$, using the previously estimated value of $\text{tr}(F(\theta_{i}))$.

### A.2 Details about Estimation of Fisher Information Trace

To check how accurate our Fisher Information trace estimate is, we ran an experiment comparing the Fisher Information trace estimated with i) a batch size of 16 once every four steps and ii) a batch size of 64 for every step (i.e., 16 times larger sample size) on CIFAR-10 Gaussian task setup.
We use ResNet-32 as the backbone and show the trace of the Fisher Information of the last layers for each block, i.e. layers 8, 16, 24, and 32.
From the result in Fig. 5, we observe that the estimation with i) a batch size of 16 once every four steps does not deviate much from the estimation with ii) a batch size of 64 for every step, showing that our estimation is reasonably accurate.

Figure: Figure 5: The estimated trace of Fisher Information for layers 8, 16, 24, and 32 of ResNet-32 on CIFAR-10 Gaussian Task setup, comparing the estimation used in aL-SAR and the estimation with a 16 times bigger sample size.
Refer to caption: x7.png

### A.3 Detailed Algorithm of aL-SAR

Algorithm 1 provides a comprehensive pseudocode for the aL-SAR method.
aL-SAR has two components: similarity-aware retrieval and adaptive layer freezing.
In the algorithm box, lines 3, 6-13, and 25-26 describe the similarity-aware retrieval method, and lines 15-24 describe the adaptive layer freezing method.

Figure: Algorithm 1 adaptive Layer freezing and Similarity-Aware Retrieval (aL-SAR)

### A.4 Implementation Details

We use ResNet-32 (He et al., ) for CIFAR-10, CIFAR-100, CLEAR-10 and CLEAR-100, and use ResNet-18 as the network architecture for ImageNet-1K.
We set the training hyperparameters as follows (Prabhu et al., ; Bang et al., ; Koh et al., ).
For CIFAR-10, CIFAR-100, CLEAR-10, and ImageNet, we use batchsize of 16, 16, 16, and 256, respectively, and Adam optimizer with LR of 0.0003 for all datasets and setup.
To calculate $A_{\text{AUC}}$, we use an evaluation period of 100 samples for CIFAR-10/100 and CLEAR-10/100, and 8000 samples for ImageNet-1K.
For memory constraints, we used memory size of 7.6MB, 13.44MB, 25.12MB for CIFAR-10 and CIFAR-100, 617MB for CLEAR-10, 5.8GB for ImageNet.

For data augmentation, we apply RandAugment (Cubuk et al., ) to all CL methods.
For hyperparameters, we set all the EMA ratios required for aL-SAR to 0.01 for all datasets. For the values of $k$ and $T$ used in memory retrieval, we use $k=4$ and $T=0.125$ for all experiments.

We found the hyperparameters through a search on CIFAR-10 and applied them to other datasets and setups without further tuning. This decision was made due to the lack of access to the dataset before actual training in real-world continual learning applications, as data arrive as an online stream
Thus, we cannot perform a dataset-specific hyperparameter search. Similarly, we do not know the distribution of the data in the future, so a setup-specific (e.g., Disjoint/Gaussian task) hyperparameter search is also not possible. Therefore, a realistic scenario is performing a hyperparameter search using a known dataset and setup in a test environment, and applying it to real-world with unknown datasets and distributions. To validate our method’s ability to adapt to unknown data in this scenario, we search for hyperparameters in CIFAR-10 Gaussian task setup and apply them to all other datasets and setups.

For aL-SAR, we use memory-only training, where the training batch is retrieved from the episodic memory at every iteration. And we use the Greedy Balanced Sampling strategy (Prabhu et al., ) for memory sampling. We use $m=4$ for all datasets and setups, where $m$ refers to the batch cycles where layer freezing is not applied.

For LiDER (Bonicelli et al., ), it follows a plug-in approach, integrated with existing methods.
Based on the experimental results in the paper, the combination with X-DER showed the most promising performance.
Consequently, the results obtained by combining X-DER were reported as the results for LiDER.

### A.5 Experiment Results using Additional Memory Constraints

The results obtained using 13.46MB memory budgets in CIFAR-10 and CIFAR-100 are shown in Fig. 6.
In addition to results in the 7.6MB memory budget in Fig. 3, our method outperforms other methods in all tested memory budgets, further showing that our method is robust across various memory constraints.

Figure: Figure 6: Accuracy on the Gaussian and the Disjoint CL setup in CIFAR-10 and CIFAR-100 for various FLOPs per sample. aL-SAR outperforms all the compared CL methods. The memory budget is fixed to 7.6MB for both CIFAR-10 and CIFAR-100.
Refer to caption: x8.png

### A.6 Comparison between Adaptive Layer Freezing and Naive Layer Freezing

We compare the proposed adaptive layer freezing method with various naive freezing methods, in both Gaussian and disjoint setup in CIFAR-10.
The results are summarized in Tab.5.
Each freezing strategy chooses the number of frozen layers $n\in[0,L]$ where $L$ is the total number of layers, so that when $n\geq 1$, layer 1 to layer $n$ are frozen.
The compared freezing strategies are: random freezing ($n$ is randomly selected from $[0,n_{\text{max}}]$ every iteration for a fixed $n_{m}ax\in[0,L]$), constant freezing ($n$ is fixed initially) and linear freezing ($n$ increases linearly from 0 to $n_{\text{max}}$ for a fixed $n_{m}ax\in[0,L]$).

All layer freezing strategies contribute to reducing computational costs.
However, adaptive layer freezing has the least performance decrease.
Note that the goal of layer freezing is not to freeze as much as possible but rather to save computational costs while preserving performance.

**Table 5: Comparison between adaptive layer freezing and naive freezing in CIFAR-10. The memory budget is 7.6MB.**
| Methods | Gaussian | Disjoint |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| No Freezing | 64.60$\pm$0.83 | 72.43$\pm$0.38 | 171.94 | 79.10$\pm$0.44 | 71.77$\pm$0.57 | 171.94 |
| Random Freezing ($n_{\text{max}}=16$) | 63.14$\pm$0.51 | 70.47$\pm$1.15 | 150.56 | 77.69$\pm$0.51 | 69.30$\pm$1.77 | 150.62 |
| Random Freezing ($n_{\text{max}}=32$) | 61.79$\pm$0.54 | 69.84$\pm$0.54 | 122.99 | 77.31$\pm$0.17 | 68.89$\pm$0.64 | 120.92 |
| Constant Freezing ($n=8$) | 60.91$\pm$0.80 | 67.70$\pm$0.83 | 147.12 | 74.99$\pm$0.24 | 65.89$\pm$0.50 | 147.12 |
| Constant Freezing ($n=16$) | 53.59$\pm$0.60 | 57.31$\pm$0.90 | 109.48 | 67.64$\pm$0.61 | 55.59$\pm$0.29 | 109.48 |
| Linear Freezing ($n_{\text{max}}=16$) | 63.11$\pm$0.90 | 70.00$\pm$1.04 | 150.64 | 77.53$\pm$0.47 | 68.30$\pm$1.52 | 150.64 |
| Linear Freezing ($n_{\text{max}}=32$) | 62.06$\pm$0.90 | 66.95$\pm$2.31 | 120.83 | 75.69$\pm$0.77 | 64.49$\pm$1.22 | 120.83 |
| Adaptive Freezing (Ours) | 64.38$\pm$0.32 | 72.57$\pm$0.79 | 146.80 | 79.75$\pm$0.38 | 70.70$\pm$0.88 | 143.51 |

### A.7 Ablation Study

In addition to the ablation study in the CIFAR-10/100 Gaussian task setup in Sec. 4.3, we ablate the model to investigate the benefit of each of the proposed components in CIFAR-10/100 Disjoint task setup and summarize the results in Tab. 6.

Summing up the effect of the two components, our method outperforms the baseline while using fewer FLOPs than the baseline, each by a noticeable margin.

**Table 6: Benefits of the proposed components of our method in CIFAR-10 and CIFAR-100 for disjoint task setup. SAR refers to our proposed Similarity-Aware Retrieval method. The memory budget is 7.6MB for CIFAR-10 and 13.44MB for CIFAR-100. CIFAR-10 We train for 1 iter per sample for CIFAR-10 and 1.5 iter per sample for CIFAR-100.**
| Methods | CIFAR-10 | CIFAR-100 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| Vanilla | 77.10$\pm$0.58 | 70.26$\pm$ 0.91 | 163.73 | 43.59$\pm$0.86 | 38.64$\pm$0.35 | 245.85 |
| + Freezing | 76.98$\pm$0.15 | 70.58$\pm$0.63 | 141.50 | 43.20$\pm$0.95 | 38.44$\pm$0.27 | 225.92 |
| + SAR | 79.10$\pm$0.44 | 71.77$\pm$0.57 | 171.94 | 45.61$\pm$1.06 | 39.68$\pm$0.66 | 257.91 |
| + SAR & Freezing (aL-SAR) | 79.75$\pm$0.38 | 70.70$\pm$0.88 | 143.51 | 45.00$\pm$1.28 | 39.39$\pm$0.62 | 228.14 |

### A.8 Ablation Study of Similarity-Aware-Retrieval (SAR)

We report the ablation results on the sub-components of Similarity-Aware Retrieval (SAR) in the Tab. 7 and Tab. 8.
Applying only the use-frequency leads to lower performance than random retrieval, since the samples from old tasks which accumulated high use frequencies in the past will be rarely used, causing severe forgetting on past tasks. Applying discounted use-frequency performs marginally better than random retrieval. Considering class similarity with effective use-frequnecy (i.e., Similarity-Aware Retrieval) further improves performance and outperforms random retrieval.
For the ablation study, memory budget is set to 7.6MB for CIFAR-10 and 13.44MB for CIFAR-100. CIFAR-10 We train for 1 iter per sample for CIFAR-10 and 1.5 iter per sample for CIFAR-100.

**Table 7: Ablation study of Similarity-Aware Retrieval on Continuous setup. ‘Vanilla’ is a simple replay-based method that trains on randomly retrieved batches from a balanced reservoir memory.**
| Methods | CIFAR-10 | CIFAR-100 |  |  |
| --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ |  |
| Vanilla | 60.76$\pm$0.11 | 70.08$\pm$0.97 | 31.97$\pm$0.89 | 37.80$\pm$1.30 |
| (+) use-frequency | 56.15$\pm$0.43 | 69.40$\pm$0.38 | 30.74$\pm$1.09 | 36.51$\pm$1.32 |
| (+) discounted-use-frequency | 62.72$\pm$0.18 | 71.10$\pm$0.53 | 33.76$\pm$0.67 | 39.29$\pm$0.98 |
| (+) effective-use-frequency (SAR) | 64.60$\pm$0.83 | 72.43$\pm$0.38 | 37.60$\pm$0.40 | 42.69$\pm$0.18 |

**Table 8: Ablation study of Similarity-Aware Retrieval on Disjoint setup. ‘Vanilla’ is a simple replay-based method that trains on randomly retrieved batches from a balanced reservoir memory.**
| Methods | CIFAR-10 | CIFAR-100 |  |  |
| --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ |  |
| Vanilla | 77.10$\pm$0.58 | 70.29$\pm$0.91 | 43.59$\pm$0.86 | 38.64$\pm$0.35 |
| (+) use-frequency | 75.40$\pm$0.36 | 69.77$\pm$1.16 | 42.11$\pm$0.81 | 37.83$\pm$0.45 |
| (+) discounted-use-frequency | 77.59$\pm$0.50 | 71.31$\pm$1.44 | 44.45$\pm$0.85 | 39.38$\pm$0.72 |
| (+) effective-use-frequency (SAR) | 79.10$\pm$0.44 | 71.77$\pm$0.57 | 45.61$\pm$1.06 | 39.68$\pm$0.66 |

### A.9 Effect of Layer Freezing in aL-SAR

Figure: Figure 7: The effect of using the adaptive freezing in aL-SAR, when training 1 iteration per sample in CIFAR-10 Gaussian task setup. The black dotted line is the linear trends from the first 10,000 samples
Refer to caption: x10.png

In Fig. 7, we present the amount of FLOPs adaptive freezing method saves, under the same number of iterations.
Our results demonstrate that adaptive freezing can save a substantial amount of training FLOPs up to 22% while maintaining accuracy.
Also, considering the FLOPs trends during the first 10,000 samples, represented as the black dotted line, we observe that the ratio of saved FLOPs increases with training progress.
This shows that our freezing scheme can capture the training progress, enabling more layers to be frozen once sufficient information has been learned.
These observations align with the trends identified in Sec. 4.3 that the ratio of saved FLOPs increases as the total computational budget increases.

### A.10 Detailed Distribution of BFC across Layers

We plot the BFC values for selected layers and how they evolve during training in Fig. 8.
Note that the figure is smoothed to show overall trends of BFC, and the actual values fluctuate depending on the input batch.
We observe that (1) BFC values tend to stay negative on average.
It indicates that freezing is not beneficial for most batches, because CL imposes a small number of training iterations and continuous streaming of new samples.
However, for input batches with little information, BFC temporarily becomes positive to allow freezing.
(2) At the task boundaries, we observe a sharp drop of BFC values, which means that the layers are less likely to be frozen so that they can learn from the new, previously unseen data.
This shows that BFC correctly handles the shift in informational contents, even though our method does not use any task boundary information.
(3) BFC of earlier layers increases as training progresses, and no longer shows significant drop of BFC values at the task boundaries.
We believe this is because the earlier layers learn low-level features that are shared across different tasks.

Figure: Figure 8: BFC of layer 10, 20, and 30 in ResNet-32 on CIFAR-10 disjoint setup. The red dotted lines indicate the task boundaries.
Refer to caption: x11.png

### A.11 Experimental Results on Tasks from Various Datasets

We compare aL-SAR with baselines on 5-datasets (Wang et al., ), which consist of five distinct datasets, to demonstrate its generalizability to the task of continual learning.
We report the results in Fig. 9.
In the figure, we can see that the x-coordinates of the aL-SAR points are shifted to the left compared to the x-coordinates of the points from other baselines, indicating that less computation was used due to freezing.
Note that as the training is done with a higher computational budget, the amount of information that can be learned from the training data relative to the computation decreases. Therefore, the freezing rate increases. In the rightmost point, it uses approximately 20% less computational cost while achieving better performance than other baselines.

Figure: Figure 9: Comparison of baselines in 5-datasets.
Refer to caption: x12.png

### A.12 Details on Training Multi-Modal Large Language Models

#### Datasets.

Beyond the class-incremental learning (CIL) setup, we extend aL to the multi-modal concept-incremental task (MCIL). Specifically, we evaluate the Bongard-HOI (Jiang et al., ) and Bongard-OpenWorld (Wu et al., ) benchmarks, which feature multiple ’concepts’ (e.g., *ride a bike*, *The top of a snow-covered mountain*) in each benchmark. We split the concepts into 5 disjoint tasks for the MCIL setup.
These benchmarks leverage two key attributes of traditional Bongard problems (Depeweg et al., ): (1) the capacity for few-shot concept learning and (2) reasoning that is dependent on context.
The former entails the ability to derive visual concepts from a limited number of examples, while the latter suggests that the classification of a query image can differ based on the context provided (i.e., the positive and negative support sets).
In the Bongard problem, with a positive support set and a negative support set, we tackle two specific tasks: (1) *Which concept is exlusively represented by the positive support set?* and (2) *For a given query image, does it belong to the positive or negative support set?*
We denote these tasks as CA (Concept Answering) and P/N, respectively.

#### Metrics.

In the P/N task, the answers are either ‘Positive’ or ‘Negative’, so we utilize accuracy as the evaluation metric. Conversely, for the CA tasks, we compare the ground truth sentences with the output sentences generated by the MLLM model. To assess this, we employ CIDEr (Vedantam et al., ), which evaluates how closely a predicted sentence aligns with a set of ground-truth sentences.
Similar to class-incremental tasks, we also measure the area under the curve (AUC) performance and the final performance within this multi-modal framework.

#### Implementation Details.

For the MCIL setup, we set the episodic memory size to 500 and use the Adam optimizer with a learning rate of $5\times 10^{-5}$ and a Constant LR scheduler. The number of images in the support set is set to 2 and 3 images for each positive and negative support set in Bongard-HOI and Bongard-OpenWorld, respectively.
We use the LLaVA-1.5-7B model and train it on NVIDIA A100 80GB GPUs.

**Table 9: The effect of adaptive freezing on MLLM training. We use LLaVA-1.5-7B model with LoRA.**
| Methods | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |
| --- | --- | --- | --- |
| No freezing | $26.43\pm 1.32$ | $\textbf{23.86}\pm\textbf{1.49}$ | 1478.78 |
| aL (Ours) | $\textbf{27.48}\pm\textbf{2.90}$ | $23.76\pm 1.79$ | 1308.72 (-11.5%) |

In Tab. 3 in Sec. 4.2, we empirically demonstrate the effectiveness of aL in the Bongard-HOI-P/N and Bongard-OpenWorld-P/N tasks.
Here, we apply aL to the Bongard-OpenWorld-CA task and summarize the results in Tab. 9.
Similar to Bongard-HOI-P/N and Bonagd-OpenWorld-P/N tasks, aL effectively reduces training costs.
Since our proposed adaptive layer freezing method is independent of downstream tasks, model architecture, and task identity information, we expect it can be applied to a variety of tasks and architectures.

### A.13 Comparison between aL and Other Freezing Methods

In addition to the quantitative comparison of aL with freezing methods in the continuous setup, we also compare them in the disjoint setup.
The results are summarized in Tab. 10.
Similar to the continuous setup, aL effectively reduces the training FLOPs in disjoint setup while maintaining performance.
In contrast, REMIND (Hayes et al., ), PTLF (Yang et al., ), and EGERIA (Wang et al., ) result in negligible reductions in training costs, indicating less freezing.
The reasons are as follows:
(1) EGERIA freezes layers where the difference between the outputs of the current model and the reference model has been marginal over recent $n$ iterations.
The reference model is periodically updated using the training model to evaluate layer plasticity.
This approach can be effective in joint training scenarios, as the training loss typically converges to zero over time, causing the layers to stabilize.
However, in online CL, training a current model with new data can cause the model to diverge from the reference model, resulting in less effective layer freezing.
(2) PTLF freezes layers with top-$k$ task correlation ratio.
However, since it freezes intermediate layers rather than layers 1 to $n$, it cannot fully reduce the backward FLOPs of the frozen layers as the gradients for layer input should still be calculated for backpropagation.

**Table 10: Comparison between our proposed adaptive layer freezing and other freezing methods. We compare them in both the CIFAR-10 disjoint setup and the CIFAR-100 disjoint setup. We train for 1 iter per sample for CIFAR-10 and 3 iter per sample for CIFAR-100.**
| Methods | CIFAR-10 | CIFAR-100 |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| No freezing | $79.10\pm 0.44$ | $\textbf{71.77}\pm\textbf{0.57}$ | 171.94 | $\textbf{53.38}\pm\textbf{1.13}$ | $\textbf{49.43}\pm\textbf{0.72}$ | 515.82 |
| REMIND | $73.20\pm 0.67$ | $62.95\pm 0.87$ | 151.31 (-12.0%) | $48.14\pm 1.30$ | $41.28\pm 0.14$ | 453.92 (-12.0%) |
| PTLF | $79.29\pm 0.36$ | $70.50\pm 0.43$ | 164.38 (-4.4%) | $53.22\pm 1.07$ | $49.25\pm 0.65$ | 497.77 (-3.5%) |
| EGERIA | $78.70\pm 0.54$ | $\underline{70.72\pm 0.68}$ | 169.52 (-1.4%) | $52.95\pm 0.77$ | $48.88\pm 0.46$ | 502.41 (-2.6%) |
| aL (Ours) | $\textbf{79.75}\pm\textbf{0.38}$ | $\underline{70.70\pm 0.88}$ | 143.51 (-16.5%) | $53.33\pm 1.21$ | $48.91\pm 0.87$ | 425.65 (-17.5%) |

### A.14 Applying Adaptive Layer Freezing to Attention-based Model

We investigate the effect of the proposed adaptive layer freezing not only in ResNet but also in attention-based models such as the Vision Transformer(ViT) (Dosovitskiy et al., ).We compare the freezing effects for both ViT-base and ViT-large models, considering those pretrained on ImageNet-1K and those trained from scratch. The results are summarized in Tab.11 and Tab. 12, respectively.

When using a pretrained model, the adaptive layer freezing reduces the computational cost by nearly 15% with minimal impact on $A_{\text{AUC}}$ and $A_{last}$, compared to the Vanilla training without freezing.
Since pretrained models have already been sufficiently trained on a large dataset, the amount of information that the model will learn from the training data may be relatively small compared to training from scratch.
Thus, it leads to the freezing of many layers by adaptive layer freezing.
This not only reduces computational costs but also ensures high performance, since the model is updated only in truly informative batches, thus preserving the advantages of pretrained initialization.

In the case of training from scratch, the decrease in TFLOPs is significantly small compared to using a pretrained model, which implies that the layers did not freeze much.
This is due to the large model capacity of ViT and the small number of training iterations in online CL, which leads to a severe underfitting of the model when training from scratch.
Specifically, Tab. 11 shows that when training from scratch, the accuracy only reaches around 30% even at the end of the training ($A_{last}$).
Thus, since the model is not sufficiently trained yet, the adaptive layer freezing scheme tends to freeze fewer layers so that the model can learn more information.
This shows that the proposed adaptive freezing method can indeed provide a reasonable freezing strategy.

**Table 11: Effect of layer freezing in ViT-base. We used CIFAR-10 and CIFAR-100 as the dataset, Gaussian Setup for setup. The memory budget is 334.76MB for both CIFAR-10 and CIFAR-100.**
| Methods | CIFAR10 | CIFAR100 |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Pretrained | From Scratch | Pretrained | From Scratch |  |  |  |  |  |  |  |  |  |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| Vanilla | 57.85$\pm$1.16 | 61.43$\pm$0.68 | 4,044.25 | 33.13$\pm$3.15 | 28.58$\pm$4.64 | 4,044.25 | 40.43$\pm$0.31 | 45.03$\pm$0.06 | 6066.4 | 14.95$\pm$5.83 | 15.61$\pm$7.58 | 6066.4 |
| + Adaptive Freezing | 58.70$\pm$1.58 | 60.73$\pm$1.52 | 3,466.42 | 33.34$\pm$3.44 | 30.44$\pm$6.22 | 3,926.65 | 41.25$\pm$2.95 | 46.58$\pm$1.71 | 5224.05 | 21.13$\pm$0.74 | 24.78$\pm$0.21 | 5828.75 |

**Table 12: Effect of layer freezing in ViT-Large. We used CIFAR-10 and CIFAR-100 as the dataset, Gaussian Setup for setup. The memory budget is 1.20GB for both CIFAR-10 and CIFAR-100.**
| Methods | Disjoint | Gaussian |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Pretrained | From Scratch | Pretrained | From Scratch |  |  |  |  |  |  |  |  |  |
| $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\ \uparrow$ | $A_{\text{last}}\ \uparrow$ | TFLOPs $\leavevmode\nobreak\ \downarrow$ |  |
| Vanilla | 69.96$\pm$3.58 | 55.03$\pm$4.46 | 14,316.37 | 41.27$\pm$0.79 | 19.52$\pm$1.47 | 14,316.37 | 58.82$\pm$2.17 | 60.11$\pm$4.81 | 14,3163.73 | 28.05$\pm$1.74 | 24.17$\pm$2.69 | 14,316.37 |
| + Adaptive Freezing | 70.78$\pm$4.70 | 58.34$\pm$4.06 | 11,002.60 | 41.63$\pm$1.09 | 21.19$\pm$1.90 | 13,802.64 | 64.15$\pm$2.70 | 73.12$\pm$0.18 | 11,306.60 | 28.93$\pm$2.08 | 24.34$\pm$1.72 | 14,017.93 |

### A.15 Applying aL-SAR across Diverse Network Architectures

aL-SAR can be applied to any feedforward neural network, as long as layers can be defined, including CNNs and Vision Transformers (Dosovitskiy et al., ), as shown in Sec. A.14.
Note that since our layer freezing methods require evaluating the information gained and FLOPs used by individual layers, a network should be dissected into layers.

**Table 13: Comparison between Baseline and aL-SAR on Disjoint and Gaussian in CIFAR-10 and CIFAR-100 with naive 8-CNN layers. The baseline refers to removing the two components of aL-SAR: similarity-aware retrieval and adaptive layer freezing.**
| Model | CIFAR-10 | CIFAR-100 |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Disjoint | Gaussian | Disjoint | Gaussian |  |  |  |  |  |  |  |  |  |
| $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ |  |
| Baseline | 78.11$\pm$0.18 | 73.44$\pm$0.91 | 506.54 | 58.95$\pm$0.11 | 74.59$\pm$0.27 | 506.54 | 45.37$\pm$1.14 | 35.68$\pm$0.35 | 759.81 | 32.91$\pm$0.64 | 34.86$\pm$0.68 | 759.81 |
| aL-SAR (Ours) | 81.25$\pm$0.23 | 75.28$\pm$0.29 | 418.19 | 66.44$\pm$0.18 | 77.08$\pm$1.09 | 437.34 | 50.15$\pm$0.92 | 38.57$\pm$0.55 | 647.77 | 41.88$\pm$0.58 | 41.65$\pm$1.02 | 668.11 |

**Table 14: Comparison between Baseline and aL-SAR on Disjoint and Gaussian in CIFAR-10 and CIFAR-100 with naive 16-CNN layers. The baseline refers to removing the two components of aL-SAR: similarity-aware retrieval and adaptive layer freezing.**
| Model | CIFAR-10 | CIFAR-100 |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Disjoint | Gaussian | Disjoint | Gaussian |  |  |  |  |  |  |  |  |  |
| $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ |  |
| Baseline | 75.45$\pm$0.86 | 72.09$\pm$0.84 | 1236.51 | 54.59$\pm$0.62 | 71.43$\pm$0.46 | 1236.51 | 40.86$\pm$1.62 | 33.56$\pm$0.98 | 1854.78 | 28.49$\pm$0.38 | 32.01$\pm$0.16 | 1854.78 |
| aL-SAR (Ours) | 79.43$\pm$0.34 | 73.62$\pm$1.31 | 1025.14 | 62.71$\pm$0.73 | 74.62$\pm$2.06 | 1082.22 | 46.22$\pm$1.38 | 39.70$\pm$0.13 | 1631.72 | 38.36$\pm$1.04 | 41.35$\pm$1.51 | 1679.66 |

**Table 15: Comparison between Baseline and aL-SAR on Disjoint and Gaussian in CIFAR-10 and CIFAR-100 with naive 32-CNN layers. The baseline refers to removing the two components of aL-SAR: similarity-aware retrieval and adaptive layer freezing.**
| Model | CIFAR-10 | CIFAR-100 |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Disjoint | Gaussian | Disjoint | Gaussian |  |  |  |  |  |  |  |  |  |
| $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ | $A_{\text{AUC}}\leavevmode\nobreak\ \uparrow$ | $A_{\text{last}}\leavevmode\nobreak\ \uparrow$ | $\text{TFLOPs}\leavevmode\nobreak\ \downarrow$ |  |
| Baseline | 68.28$\pm$0.95 | 65.18$\pm$1.66 | 2096.46 | 46.00$\pm$1.55 | 60.30$\pm$2.39 | 2696.46 | 27.81$\pm$0.30 | 23.11$\pm$0.78 | 4044.71 | 17.69$\pm$1.16 | 21.69$\pm$1.43 | 4044.71 |
| aL-SAR (Ours) | 73.42$\pm$0.41 | 66.51$\pm$0.23 | 2304.73 | 53.21$\pm$3.01 | 66.97$\pm$3.87 | 2495.75 | 29.80$\pm$0.96 | 26.05$\pm$1.00 | 3910.97 | 24.68$\pm$2.10 | 29.69$\pm$2.65 | 3940.56 |

To analyze the effect of the number of layers in aL-SAR, we first conduct experiments with ResNet-20 and ResNet-56 on CIFAR-10 and CIFAR-100, in addition to ResNet-32 and ResNet-18 as used in the main results.
As shown in Fig. 10 and Fig. 11, aL-SAR consistently outperforms baseline, which refers to removing two components of aL-SAR: adaptive layer freezing and similarity-aware retrieval, irrespective of the number of layers.
We also compare aL-SAR with other CL baselines and summarize
the results in Fig.12 and Fig. 13, respectively.
aL-SAR consistently outperforms other methods in both shallower and deeper networks, showing that aL-SAR is robust in various model sizes.

Moreover, we perform additional experiments on the naive CNN without skip connection, i.e., consisting only of convolution, batch normalization, activation, and fully connected layers.
We report the result in Tab. 13, Table Tab. 14, and Tab. 15 for 8-layer, 16-layer, and 32-layer CNNs, respectively.
In the table, aL-SAR improves performance and reduces computational cost compared to baseline also when using a simple CNN, regardless of the number of layers.
Note that a deeper CNN shows lower performance due to the absence of skip connections, as reported in [1] (ResNet).

However, our proposed adaptive layer freezing cannot apply to recurrent neural networks (Sherstinsky, ) since the gradient of a layer affects not only the preceding layers but also the subsequent layers (Rotman & Wolf, ), while in the feedforward network, the gradient of a layer influences only the preceding layers.

Figure: Figure 10: Comparison of the baseline and aL-SAR on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 with ResNet-20. The baseline refers to removing the two components of aL-SAR, i.e., adaptive layer freezing and similarity-aware retrieval, from aL-SAR itself.
Refer to caption: x13.png

Figure: Figure 11: Comparison of the baseline and aL-SAR Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 with ResNet-56. The baseline refers to removing the two components of aL-SAR, i.e., adaptive layer freezing and similarity-aware retrieval, from aL-SAR itself.
Refer to caption: x15.png

Figure: Figure 12: Comparison of CL methods on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 with ResNet-20.
Refer to caption: x17.png

Figure: Figure 13: Comparison of CL methods on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 with ResNet-56.
Refer to caption: x19.png

### A.16 Details about A AUC subscript 𝐴 AUC A_{\text{AUC}} italic_A start_POSTSUBSCRIPT AUC end_POSTSUBSCRIPT

Recent studies (Pellegrini et al., ; Caccia et al., ; Banerjee et al., ; Ghunaim et al., ) suggest that having good inference performance at any intermediate time points during training is important for CL. To evaluate intermediate performance during training, (Koh et al., ) proposed $A_{\text{AUC}}$, which measures the area under the curve of average accuracy.
In contrast to $A_{last}$ or $A_{avg}$ which measures performance only at the end of the task (i.e., after sufficient training), $A_{\text{AUC}}$ consistently measures performance over the course of training. If two methods reach the same accuracy at the end of a task, but one method converges faster than the other, their $A_{last}$ and $A_{avg}$ would be equal, but the faster model would show higher $A_{\text{AUC}}$. Thus, how fast the model adapts to the new task is reflected in $A_{\text{AUC}}$.

### A.17 Detailed Information on the Calculation of 𝒮 y i , y j subscript 𝒮 subscript 𝑦 𝑖 subscript 𝑦 𝑗 \mathcal{S}_{y_{i},y_{j}} caligraphic_S start_POSTSUBSCRIPT italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , italic_y start_POSTSUBSCRIPT italic_j end_POSTSUBSCRIPT end_POSTSUBSCRIPT

We calculate the cosine similarity of the gradients between all pairs of samples in the training batch and update the EMA estimate of $S_{y_{i}y_{j}}$ using it.
To further reduce the computational cost of calculating similarity, we use only $0.05\%$ of the model parameters for the calculation of similarity, since the gradient distribution of the subset of randomly selected weights is similar to the gradient of the entire weight set (Li et al., ).

### A.18 Details about Measuring FLOPs

With the exception of the simplest baseline, ER, the other baseline methods involve additional computational costs. REMIND, MIR, OCS, X-DER, and MEMO require additional forward/backward processes of the model, resulting in a notable increase in the relative FLOPs compared to ER. On the contrary, for DER, EWC, and aL-SAR, additional computational cost is neglible as they do not involve model forward and backward processes i.e., the relative FLOPs is approximately 1.
We compare the types of additional computational costs and relative flops compared to ER (Rolnick et al., ) for each method in Tab. 16.

Although aL-SAR requires various types of computation, these amounts to significantly fewer operations compared to the costs incurred during model forward and backward passes.
In addition, since aL-SAR freezes layers depending on the input batch, we provided a range for backward FLOPs.
Note that Remind incurs additional costs during base initialization to convert all memory data into features and perform product quantization.
However, since this process occurs only once during the model training process and not in every iteration, it is not included in the table.

**Table 16: Details of the additional computational budget. To measure forward/backward FLOPs of the model, we use ptflops(^2^22[https://github.com/sovrasov/flops-counter.pytorch](https://github.com/sovrasov/flops-counter.pytorch)), which is a widely used Python library to calculate FLOPs. FLOPs from other operations were manually calculated.**
| Methods | Type of Computation | FLOPs formulation | FLOPs/sample (TFLOPs) | Total FLOPs/sample (TFLOPs) | Relative FLOPs to ER |
| --- | --- | --- | --- | --- | --- |
| ER | Forward FLOPs | Measured by ptflops | 0.57 | 1.71 | 1 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| DER | Forward FLOPs | Measured by ptflops | 0.57 | 1.71 | $\approx$ 1 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Calculation of Distillation Loss | 2 $\times$ ($\#$ classes) | $1.0\times 10^{-11}$ |  |  |  |
| X-DER | Forward FLOPs | Measured by ptflops | 0.57 | 3.99 | 2.33 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Forwarding Inputs with Different<br>Augmentations for Contrastive Learning | 4 $\times$ (Forward FLOPs) | 2.28 |  |  |  |
| OCS | Forward FLOPs | Measured by ptflops | 0.57 | 3.42 | $\approx 2$ |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Coreset sampling | (Forward FLOPs + Backward FLOPs) | 1.71 |  |  |  |
| ER-MIR | Forward FLOPs | Measured by ptflops | 0.57 | 7.05 | 4.12 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Forward FLOPs for Candidates | $\frac{(\#\text{candidates})}{(\text{batchsize})}\times$ 0.57 | 1.78 |  |  |  |
| Backward FLOPs for Candidates | $\frac{(\#\text{candidates})}{(\text{batchsize})}\times$ 1.14 | 3.56 |  |  |  |
| MEMO | Forward FLOPs | Measured by ptflops | 0.57 | 2.55 | 1.49 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Forward cost for Expanded Network | ($\#$ of tasks) $\times$ 0.03 | 0.15 |  |  |  |
| Backward cost for Expanded Network | ($\#$ of tasks) $\times$ 0.07 | 0.35 |  |  |  |
| LiDER | Forward FLOPs | Measured by ptflops | 0.57 | 3.99 | 2.33 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Forwarding Inputs with Different-<br>Augmentations for Contrastive Learning | 4 $\times$ (Forward FLOPs) | 2.28 |  |  |  |
| Calculation of Lipschitz Constant | (batchsize) $\times$ f.shape[0] * f.shape[0]<br>$\times$ (2 $\times$ f.shape[1] - 1) | 4.19 $\times 10^{-6}$ |  |  |  |
| CCL-DC | Forward FLOPs | Measured by ptflops | 0.57 | 7.98 | 4.66 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Forwarding Inputs with Different-<br>Augmentations for Collaborative Learning | 9 $\times$ (Forward FLOPs) + (Backward FLOPs) | 6.27 |  |  |  |
| CAMA | Forward FLOPs | Measured by ptflops | 0.57 | 1.71 | $\approx$ 1 |
| Backward FLOPs | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Calculation of Distillation Loss | 2 $\times$ ($\#$ classes) | $1.0\times 10^{-11}$ |  |  |  |
| Logit Updating | 3 $\times$ ($\#$ classes) | $1.5\times 10^{-11}$ |  |  |  |
| REMIND | Forward FLOPs (before base initialization) | Measured by ptflops | 0.57 | 1.71 | 1 |
| Backward FLOPs (before base initialization) | Measured by ptflops | 1.14 |  |  |  |
| Model Updates (Adam - before base initialization) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Forward FLOPs (after base initialization) | 0.78 $\times$ (Forward FLOPs for whole model) | 0.44 | 1.32 | 0.7 |  |
| Backward FLOPs (after base initialization) | 0.78 $\times$ (Backward FLOPs for whole model) | 0.88 |  |  |  |
| Model Updates (Adam - after base initialization) | 0.78 $\times$ (Model updates for whole model) | 5.52 $\times 10^{-6}$ |  |  |  |
| aL-SAR | Forward FLOPs | Measured by ptflops | 0.57 | 0.57 $\sim$ 1.71 | 0.33 $\sim$ 1 |
| Backward FLOPs | Measured by ptflops (depends on freezing) | $8.2\times 10^{-8}\sim 1.14$ |  |  |  |
| Model Updates (Adam) | 12 $\times$ ($\#$ parameters) | 5.52 $\times 10^{-6}$ |  |  |  |
| Calculation of Class-wise similarity | $(5\times(\#\leavevmode\nobreak\ \text{parameters})\times 0.0005+3)$<br>$\times(\text{batchsize}$ | $5.5\times 10^{-8}$ |  |  |  |
| Calculation of Fisher Information | $2\times(\#\leavevmode\nobreak\ \text{parameters})+3\times(\#\leavevmode<br>\nobreak\ \text{layers})$ | $9.2\times 10^{-7}$ |  |  |  |
| Calculation of BFC | $(\#\leavevmode\nobreak\ \text{layers})\times 3+\text{len}(x_{L})\times 2$ | $1.1\times 10^{-9}$ |  |  |  |
| Calculation of Retrieval Probability | $4\times|M|$ + $(\#\text{ classes})^{2}+(\#\text{ classes})$ | 8.11 $\times\leavevmode\nobreak\ 10^{-9}$ |  |  |  |
| Frequency Update | $(\text{batchsize})\times 2+(\#\text{ classes})+|M|$ | 2.0 $\times 10^{-9}$ |  |  |  |

### A.19 Details about the Memory bBudget in Total-constrained CL

The memory budget is allocated to episodic memory, model parameters, and additional memory costs specific to each CL algorithm, such as classwise similarities and logits.
In this section, $\mathcal{B}$ denotes the additional memory budget, S($\lvert\mathcal{B}\lvert$) denotes the size of the additional memory budget (in MB), $\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{\mbox{
\char 130\relax}}{\mbox{\char 130\relax}}}$ denotes episodic memory, and $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{
\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ represents the number of stored instances in $\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{\mbox{
\char 130\relax}}{\mbox{\char 130\relax}}}$.

In our total-constrained setup, the memory budget is restricted to the cost of storing 7.6MB, 13.44MB and 25.12 MB in CIFAR-10/100. Since storing the ResNet-32 model parameters requires memory cost equivalent to saving 603 instances of CIFAR-100 images (463,504 floats × 4 bytes/float ÷ (3 × 32 × 32) bytes/image $\approx$ 603 instances), for methods that store the model for distillation or regularization, we subtract the memory cost of the model parameters from the episodic memory size (Zhou et al., ).
In ImageNet and CLEAR-10, we use the ResNet-18 model and apply the same policy of subtracting model parameters and logits from the memory budget as mentioned above.

ER does not require additional memory beyond episodic memory.
Similarly, MIR (Aljundi et al., ) and ASER (Shim et al., ) do not require additional memory despite being computationally heavy.

On the contrary, EWC (Kirkpatrick et al., ) requires storing the previous model parameters and the parameter-wise Fisher Information(FI) for all parameters.
Therefore, we subtract the memory cost of storing two models from the episodic memory size. Similarly, BiC (Wu et al., ) also stores the previous model for distillation, $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{
\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ was reduced as much as the size of the model.
For example, with a total memory budget of 7.6MB and a ResNet-32 model type in CIFAR-100, ER can store up to 2000 instances in $\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{\mbox{
\char 130\relax}}{\mbox{\char 130\relax}}}$, while EWC is limited to storing only 794 (= 2000 - 2$\times$603) instances.
Similarly, BiC can store only 1397 (= 2000 - 603) instances in $\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{\mbox{
\char 130\relax}}{\mbox{\char 130\relax}}}$.

Some methods incur additional memory costs other than episodic memory or model parameters.
We handle such costs in a similar way by reducing the episodic memory size by the number of samples equivalent to the additional memory cost.
For example, DER (Buzzega et al., ) uses the previous logits of the samples for distillation, so we subtract the cost of storing the logits from the episodic memory size.
More specifically, DER needs additional storage which size is $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{
\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert\times d_{l}\times$ 4 bytes/float, where $d_{l}$ denotes logit dimension, which is 100 in CIFAR-100 and 10 in CIFAR-10.

aL-SAR stores similarities between classes, the training frequency of each sample, and the trace of FIM for each layer. AR needs 400 bytes = 4 bytes/float $\times 10^{2}$ for saving class-wise similarities, 4 bytes/int $\times\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax
}}{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ for saving frequency of each sample, and 4 bytes/float $\times n_{l}$ for saving trace of FIM for each layer, where $n_{l}$ is total number of layers.
However, such additional memory cost is negligible compared to episodic memory or model parameters (only 0.1% of memory budget).
We summarize implementation details of the total memory budget for each dataset in Tab.17, Tab.18, Tab.19,Tab.20, and Tab.21.

**Table 17: Implementation details of total memory budget=7.6MB in CIFAR-10**
| Methods | $\mathcal{B}$ Type | $S(\lvert\mathcal{B}\lvert)$ | $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ | S$(\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert)$ | Model Type | # Parameters | Model Size |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ER | - | - | 2,000 | 5.85MB | Resnet32 | 0.46M | 1.76MB |
| REMIND | Feature replay | 5.85MB | - | - | Resnet32 | 0.46M | 1.76MB |
| DER | Logits | 0.08MB | 1,974 | 5.77MB | Resnet32 | 0.46M | 1.76MB |
| ER-MIR | - | - | 2,000 | 5.85MB | Resnet32 | 0.46M | 1.76MB |
| EWC | FI & Previous Model | 3.52MB | 794 | 2.33MB | Resnet32 | 0.46M | 1.76MB |
| OCS | - | - | 2,000 | 5.85MB | Resnet32 | 0.46M | 1.76MB |
| X-DER | Logits | 0.08MB | 1,974 | 5.77MB | Resnet32 | 0.46M | 1.76MB |
| LiDER | Logits | 0.08MB | 1,974 | 5.77MB | Resnet32 | 0.46M | 1.76MB |
| MEMO | Expanded Network | 3.52MB | 794 | 2.33MB | Resnet32 | 0.46M | 1.76MB |
| CAMA | Logits | 0.08MB | 1,974 | 5.77MB | Resnet32 | 0.46M | 1.76MB |
| CCL-DC | Teacher model for<br>collaborative learning | 1.76MB | 1397 | 4.08MB | Resnet32 | 0.46M | 1.76MB |
| aL-SAR | Class-wise similarity &<br>frequency of each sample | $8.52$KB | 1,997 | 5.84MB | Resnet32 | 0.46M | 1.76MB |

**Table 18: Implementation details of total memory budget=7.6MB in CIFAR-100**
| Methods | $\mathcal{B}$ Type | $S(\lvert\mathcal{B}\lvert)$ | $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ | S$(\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert)$ | Model Type | # Parameters | Model Size |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ER | - | - | 2,000 | 5.85MB | Resnet32 | 0.46M | 1.76MB |
| REMIND | Feature replay | 5.85MB | - | - | Resnet32 | 0.46M | 1.76MB |
| DER | Logits | 0.71MB | 1,770 | 5.14MB | Resnet32 | 0.46M | 1.76MB |
| ER-MIR | - | - | 2,000 | 5.85MB | Resnet32 | 0.46M | 1.76MB |
| EWC | FI & Previous Model | 3.52MB | 794 | 2.33MB | Resnet32 | 0.46M | 1.76MB |
| OCS | - | - | 2,000 | 5.85MB | Resnet32 | 0.46M | 1.76MB |
| X-DER | Logits | 0.71MB | 1,770 | 5.14MB | Resnet32 | 0.46M | 1.76MB |
| LiDER | Logits | 0.71MB | 1,770 | 5.14MB | Resnet32 | 0.46M | 1.76MB |
| MEMO | Expanded Network | 1.33MB | 1,542 | 4.51MB | Resnet32 | 0.46M | 1.76MB |
| CAMA | Logits | 0.71MB | 1,770 | 5.14MB | Resnet32 | 0.46M | 1.76MB |
| CCL-DC | Teacher model for<br>collaborative learning | 1.76MB | 1397 | 4.08MB | Resnet32 | 0.46M | 1.76MB |
| aL-SAR | Class-wise similarity &<br>frequency of each sample | 0.05MB | 1,987 | 5.83MB | Resnet32 | 0.46M | 1.76MB |

**Table 19: Implementation details of total memory budget=574.4MB in CLEAR-10**
| Methods | $\mathcal{B}$ Type | $S(\lvert\mathcal{B}\lvert)$ | $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ | S$(\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert)$ | Model Type | # Parameters | Model Size |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ER | - | - | 4,000 | 574.0MB | Resnet32 | 0.46M | 1.76MB |
| REMIND | Feature replay | 574.0MB | - | - | Resnet32 | 0.46M | 1.76MB |
| DER | Logits | 0.1MB | 3,999 | 573.9MB | Resnet32 | 0.46M | 1.76MB |
| ER-MIR | - | - | 4,000 | 574.0MB | Resnet32 | 0.46M | 1.76MB |
| EWC | FI & Previous Model | 4.0MB | 3,972 | 570.0MB | Resnet32 | 0.46M | 1.76MB |
| OCS | - | - | 4,000 | 574.0MB | Resnet32 | 0.46M | 1.76MB |
| X-DER | Logits | 0.1MB | 3,999 | 573.9MB | Resnet32 | 0.46M | 1.76MB |
| LiDER | Logits | 0.1MB | 3,999 | 573.9MB | Resnet32 | 0.46M | 1.76MB |
| MEMO | Expanded Network | 1.4MB | 3,990 | 572.6MB | Resnet32 | 0.46M | 1.76MB |
| CAMA | Logits | 0.1MB | 3,999 | 573.9MB | Resnet32 | 0.46M | 1.76MB |
| CCL-DC | Teacher model for<br>collaborative learning | 1.76MB | 3,987 | 572.2MB | Resnet32 | 0.46M | 1.76MB |
| aL-SAR | Class-wise similarity &<br>frequency of each sample | 0.1MB | 3,999 | 573.9MB | Resnet32 | 0.46M | 1.76MB |

**Table 20: Implementation details of total memory budget=1149.8MB in CLEAR-100**
| Methods | $\mathcal{B}$ Type | $S(\lvert\mathcal{B}\lvert)$ | $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ | S$(\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert)$ | Model Type | # Parameters | Model Size |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ER | - | - | 8,000 | 1,148.0MB | Resnet32 | 0.46M | 1.76MB |
| REMIND | Feature replay | 1,148.0MB | - | - | Resnet32 | 0.46M | 1.76MB |
| DER | Logits | 3.2MB | 7,978 | 1,144.8MB | Resnet32 | 0.46M | 1.76MB |
| ER-MIR | - | - | 8,000 | 1,148.0MB | Resnet32 | 0.46M | 1.76MB |
| EWC | FI & Previous Model | 4.0MB | 7,972 | 1,144.0MB | Resnet32 | 0.46M | 1.76MB |
| OCS | - | - | 8,000 | 1,148.0MB | Resnet32 | 0.46M | 1.76MB |
| X-DER | Logits | 3.2MB | 7,978 | 1,144.8MB | Resnet32 | 0.46M | 1.76MB |
| LiDER | Logits | 3.2MB | 7,978 | 1,144.8MB | Resnet32 | 0.46M | 1.76MB |
| MEMO | Expanded Network | 1.4MB | 7,990 | 1,146.6MB | Resnet32 | 0.46M | 1.76MB |
| CAMA | Logits | 3.2MB | 7,978 | 1,144.8MB | Resnet32 | 0.46M | 1.76MB |
| CCL-DC | Teacher model for<br>collaborative learning | 1.76MB | 7,987 | 1146.2MB | Resnet32 | 0.46M | 1.76MB |
| aL-SAR | Class-wise similarity &<br>frequency of each sample | 0.1MB | 7,999 | 1,147.9MB | Resnet32 | 0.46M | 1.76MB |

**Table 21: Implementation details of total memory budget=5,782.6MB in ImageNet**
| Methods | $\mathcal{B}$ Type | $S(\lvert\mathcal{B}\lvert)$ | $\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert$ | S$(\lvert\mathord{\mathchoice{\mbox{\char 130\relax}}{\mbox{\char 130\relax}}{<br>\mbox{\char 130\relax}}{\mbox{\char 130\relax}}}\lvert)$ | Model Type | # Parameters | Model Size |
| --- | --- | --- | --- | --- | --- | --- | --- |
| ER | - | - | 40,000 | 5,740.0MB | Resnet18 | 11.17M | 42.6MB |
| REMIND | Feature replay | 5,740.0MB | - | - | Resnet18 | 11.17M | 42.6MB |
| DER | Logits | 148.6MB | 38,964 | 5,591.4MB | Resnet18 | 11.17M | 42.6MB |
| ER-MIR | - | - | 40,000 | 5,740.0MB | Resnet18 | 11.17M | 42.6MB |
| EWC | FI & Previous Model | 85.2MB | 39,406 | 5,654.8MB | Resnet18 | 11.17M | 42.6MB |
| OCS | - | - | 40,000 | 5,740.0MB | Resnet18 | 11.17M | 42.6MB |
| X-DER | Logits | 148.6MB | 38,964 | 5,591.4MB | Resnet18 | 11.17M | 42.6MB |
| LiDER | Logits | 148.6MB | 38,964 | 5,591.4MB | Resnet18 | 11.17M | 42.6MB |
| MEMO | Expanded Network | 32.0MB | 39,777 | 5,708.0MB | Resnet18 | 11.17M | 42.6MB |
| CCL-DC | Teacher model for<br>collaborative learning | 42.6MB | 39,703 | 5,697.4MB | Resnet18 | 11.17M | 42.6MB |
| aL-SAR | Class-wise similarity &<br>frequency of each sample | 3.9MB | 39,973 | 5,736.1MB | Resnet18 | 11.17M | 42.6MB |

### A.20 Experimental Results in Memory Infinite Setup

aL-SAR demonstrates even stronger performance with an unlimited memory budget setup in various benchmarks such as CIFAR-10, CIFAR-100, and ImageNet-1K, as shown in Fig. 14.
We believe it is because the effect of the retrieval strategy is more significant with unlimited memory than with limited memory.
Provided that the retrieval strategy successfully distinguishes useful samples, retrieving from the full past data (unlimited memory) is likely to yield more useful samples than retrieving only from a small portion of past data (limited memory). Thus, strong performance in the unlimited-memory setup indicates that our similarity-aware retrieval effectively distinguishes useful samples.

Interestingly, aL-SAR freezes fewer layers in the unlimited memory setup than the limited memory setup.
In the unlimited memory setup, the model can learn more knowledge from the samples stored in episodic memory than in the limited memory setup, so our adaptive freezing scheme chooses to freeze fewer layers so that the model can acquire more knowledge.
It shows that our adaptive freezing method works also in the unlimited memory setup without the need to modify.

Figure: Figure 14: $A_{last}$ and $A_{\text{AUC}}$ on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100 for a wide range of FLOPs per sample with infinite memory budget. We employ ResNet-32 for CIFAR-10/100 and ResNet-18 for ImageNet-1K as a backbone.
Refer to caption: x21.png

### A.21 Comparison with Retrieval-Based CL Methods

To train models with informative training batches, recent studies propose retrieving batches from episodic memory, such as MIR (Aljundi et al., ) and ASER (Shim et al., ).
However, calculating the loss in MIR and the adversarial shapely value in ASER requires additional forward and backward processes, leading to high computational requirements.
In contrast, our proposed similarity-aware retrieval methods only utilize the gradient vector, which can be obtained naturally during the training process, incurring no additional cost. Therefore, as depicted in Fig. 15, aL-SAR outperforms MIR and ASER under fixed computational budgets.

Figure: Figure 15: Comparison of various retrieval-based CL methods on Gaussian and Disjoint CL setup in CIFAR-10 and CIFAR-100.
Refer to caption: x24.png

### A.22 Effect of Temperature T 𝑇 T italic_T

Temperature is selected by a hyperparameter search on CIFAR-10. The lower the temperature, the more the retrieval focuses on samples with low effective use-frequency, enabling a faster gain in knowledge. However, since we need diverse samples to maintain an accurate estimate of similarity, a too low temperature would also hinder the performance. Thus, we select an adequate temperature via a hyperparameter search. We report the result of the hyperparameter search in Fig. 16. Note that while a too high or low temperature results in a diminished performance, there is a wide range of temperature values that show stable performances.

Figure: Figure 16: Experiment for the effect of temperature $T$ in the CIFAR-10 Gaussian scheduled setup. We use ResNet-32 as a backbone
Refer to caption: x26.png

### A.23 Comparison with SOTA Efficient CL Methods

We compare aL-SAR with with existing SOTA efficient CL methods and summarize the results in Fig. 17.
TriRE incorporates distillation through an Exponential Moving Average (EMA) model, necessitating extra forward computation and memory allocation for storage.
Consequently, within a total-constrained setup, TriRE retains fewer samples in episodic memory.
As a result, TriRE exhibits lower overall performance compared to aL-SAR and SparCL.
Especially, TriRE has a significantly lower $A_{\text{AUC}}$ than $A_{\text{last}}$.
This is because $A_{\text{AUC}}$ not only includes inference performance at the end of a task but also incorporates anytime inference performance. Specifically, in TriRE, the inference performance at points where all three stages of TriRE, namely Retain, Revise, and Rewind, are not completed tends to be low.
Note that both TriRE and SparCL rely on task boundary information, while aL-SAR does not depend on task boundary information, leading to practical utilization in real-world scenarios.

Figure: Figure 17: Comparison with efficient CL methods under both memory and computation constrained setup. In SparCL, the underscore represents sparsity. SparCL exhibits its best performance when the sparsity is set to 0.75.
Refer to caption: x27.png

### A.24 Toy Experiments on Similarity-Aware Retrieval

We performe several toy experiments to validate the motivations for our proposed retrieval method.
To isolate the effect of each sample, we train ResNet-20 from scratch with batch size of 1 using a subset of CIFAR-10.
First, to validate our claim *"extensively used samples provide little knowledge to the model"*, we plot (use frequency) vs (training loss decrease caused by training with the sample) in Fig. 18.
We observe that training with a frequently used sample results in a small decrease in training loss.

Next, we validate the exponential decaying model of forgetting.
To measure this, we first train the model with one sample and measure the use frequency along with the corresponding training loss.
Next, when training the model with different samples, we measure the loss of the previously trained sample.
At this point, due to the training of other samples, forgetting occurred for the sample we used for initial training, causing the training loss to increase again. We then use the measured use-frequency and corresponding training loss values, which are measured in the first stage, to reverse-transform the training loss into an effective use-frequency.
As shown in Fig. 19, as the number of training iterations for other samples increases, we observe that the training loss for previously trained samples increases, leading to a decrease in the effective use-frequency.
Note that the $y$-axis is in log scale and the plot shows a linear trend, showing that the loss increase from forgetting corresponds to exponential decay of use frequency.

Finally, we validate that cosine similarity between gradients can be used to measure the effective change in use frequency from using other samples.
Using the same setup as the previous experiment, in Fig. 20, we show a scatter plot between gradient similarity and the effective change in use-frequency obtained by the relation between the sample’s loss and use frequency.
When training with samples that have high gradient similarity with the reference samples, the effective use frequency of the reference samples increases.
On the contrary, when training with samples that have low gradient similarity, the effective use frequency of reference samples decreases.
We observe a linear relationship between gradient similarity and the change in effective use-frequency.
In other words, the change in effective use frequency can be predicted using gradient similarity.

Figure: Figure 18: Relation between a sample’s use frequency and training loss decrease from training with the sample, measured in CIFAR-10 with ResNet-20. When a sample is used more frequently in training, it has less effect on reducing training loss.
Refer to caption: x29.png

Figure: Figure 19: Plot showing the number of iterations from the point where the sample has not been used for training, and the decrease in effective use frequency corresponding to the increase in loss. It is measured in CIFAR-10 with ResNet-20.
Refer to caption: x30.png

Figure: Figure 20: The scatter plot shows the cosine similarity between the gradient of a reference sample and the gradients of other samples, along with the change in the effective use frequency of the reference sample when training with other samples. It is performed in CIFAR-10 with ResNet-20.
Refer to caption: x31.png

### A.25 Limitations and Future Work

While our method only requires negligible additional memory other than episodic memory, it does not actively optimize the memory efficiency of CL algorithms.
It is interesting to explore a method to use the limited storage budget more efficiently, e.g., storing quantized versions of models and exemplars.

### A.26 Impact Statement

This work aims to update a model in a computationally efficient and online manner without access to training samples used before.
Thus, although there is no intent from the authors, this method may exacerbate unsolved issues in deep learning such as model bias and less aligned to ethical standards.
We will take all available measures to prevent such outcomes, though that is *not* our intention *at all*.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: C1:
  - FLOPs served as a direct measure of computational demand across different models, making it possible to compare CL algorithms fairly. However, it doesn’t account for real-world runtime or memory access patterns, which vary based on hardware and software implementations. For instance, two architectures with similar FLOPs might have different inference times on the same hardware due to parallelization or memory bandwidth usage differences.
  
C2:
  - The adaptive layer freezing proposed in this paper can reduce FLOPs but adds complexity. Calculating the full Fisher Information Matrix (FIM) is computationally intensive because it involves second-order derivatives, which can be prohibitive, especially for large neural networks. The paper must explain why calculating FIM does not introduce additional computation costs.
  - Fisher Information (FI) is calculated based on the current mini-batch. However, the calculation results based on the current mini-batch may not generalize across diverse batches in continual learning. For instance, a layer might appear less informative for one mini-batch but could be essential for others due to shifting data distributions, especially under continual learning settings. Hence, the paper needs to justify this.

C3:
  - The strategy might prioritize rare samples in the dataset but may need to be more informative. This overemphasis can lead to inefficient learning, as these rare samples may not enhance generalization and introduce noise. The paper needs to justify that such a retrieval strategy would maintain the acquired knowledge during training.

**Writing Issues**

  1. Line 071: 'ImageNet)' -> 'ImageNet'.
  2. Line 087: 'ER' has no citation and explanation for the abbreviation.
  3. Fig. 1: No explanation of the y-axis.
soundness: 3
strengths: ++ Standardizing computational and memory budgets of continual learning (CL) algorithms is important for evaluating algorithm efficiency and learning system design.

++ The paper provides comprehensive and detailed experimental results.
summary: This paper proposes the method *aL-SAR*, which standardizes computational and memory budgets of continual learning (CL) algorithms based on training floating point operations (FLOPs) and total memory size in Byte to address the challenge of fair comparison in online CL algorithms. Besides, this paper introduces adaptive layer freezing to reduce computation costs by selectively freezing layers and frequency-based sampling to prioritize under-learned samples. Experiments on CIFAR-10/100 and ImageNet-1K datasets show that aL-SAR consistently outperforms existing CL methods like ER (Experience Replay), DER++, MIR (Maximally Interfered Retrieval), MEMO, REMIND, EWC, OCS, LiDER, X-DER, CCL-DC, and CAMA by up to 5-10% on average across AAUC and last accuracy, especially under stringent budget constraints.
weaknesses: -- Using FLOPs as a metric focuses on raw operations, ignoring algorithm-specific optimizations for training software and hardware systems. See C1.

-- The additional computations required to decide which layers to freeze may introduce overhead that is not directly part of the core training but still contributes to the total computational cost. See C2.

-- The retrieval strategy based on the use-frequency cannot ensure that the model does learn sufficiently by preferring less frequently used samples. See C3.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: * What is the memory selection mechanism? In other words, how memory samples are selected, and how does the memory get updated?

* Besides the similarity-aware retrieval, is there any other difference between the in-memory and new samples in the training?
soundness: 3
strengths: * The arguments about the FLOPS and memory costs are critical. To have a fair and meaningful comparison between different methods, we *need* to know the actual cost.

* The idea of having batch-wise and dynamic layer freezing makes the training more suitable for online settings.

* The authors did an excellent job of explaining the motivation and algorithm. All the different components are adequately explained and justified.
summary: This paper proposes a new algorithm for the online CL problem. The authors first explain that a fair comparison between different methods should consider the total number of FLOPs instead of the total number of iterations, as some methods can perform expensive operations each round. Also, it is essential to include all types of memory usage rather than only considering the number of examples in the episodic memory as the actual memory cost. Next they propose an efficient algorithm to address the problems in online continual learning. They use batch fisher information to find the optimal layers that can be frozen while learning from the data. Then, they introduce a new metric, 'appearance frequency,' to measure the contributions of each training example.
weaknesses: * In my opinion, the introduction lacks proper background knowledge, and it jumps to the solutions for the existing problems in online CL. I suggest adding at least one paragraph explaining the background and current state of online continual learning and the current high-level challenges that prior work has attempted to solve, so the readers are on the same page and maybe appreciate your work even more. 

* Have you considered reporting forgetting as an additional metric? This is commonly used in CL papers and could provide further insight into the efficacy of your method.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Could the adaptive layer freezing strategy be extended to handle cross-layer dependencies, potentially leading to more optimal freezing decisions?
2. Although authors have illustrated the process of calculating *Effective Use Frequency* in Sec.3.2 and Sec.A.17, it is still vague for me that how to calculate cosine similarity of the gradients between classes? In A.17, do it that using *0.05% of the model parameters* means that randomly sample params among unfrozen layers?
soundness: 3
strengths: 1. Novel framework for comparing CL methods using total resource budgets including innovative adaptive layer freezing approach based on Fisher Information and computationally efficient memory retrieval strategy.

2. Rigorous empirical validation across multiple datasets and setups containing thorough ablation studies and comparisons with state-of-the-art methods and successful application to large models (LLaVA-1.5-7B).

3. Addresses practical constraints in online CL and shows significant performance improvements while reducing computational costs.
summary: This paper addresses the challenge of fair comparison in online continual learning (CL) by proposing a total resource budget framework that considers both computational costs (FLOPs) and memory usage (bytes). The authors introduce aL-SAR, which combines two key innovations:

1. Adaptive layer freezing: A method that selectively freezes network layers based on Fisher Information to reduce computational costs while maintaining performance
2. Similarity-aware retrieval: A computationally efficient strategy for retrieving training samples based on usage frequency and gradient similarity

The method is extensively validated on CIFAR-10/100, CLEAR-10/100, and ImageNet-1K datasets, showing superior performance compared to state-of-the-art methods under the same resource constraints.
weaknesses: 1. Using the layer freezing technque for recuding the model params would always block the shallow layers (i.e., layer 1-n) and their representation would be outdated. This would be problematic for frequent distribution shifts.
   
2. The paper's adaptive layer freezing strategy treats layers somewhat independently when making freezing decision. However, neural networks typically have complex inter-layer relationships that this approach might not fully capture.
   
3. The method doesn't address catastrophic forgetting directly, instead focusing on resource efficiency. It's unclear how the method balances the trade-off between resource efficiency and forgetting prevention.
   
4. The computational overhead of maintaining class-wise gradient similarities might become prohibitive for problems with a large number of classes or frequent distribution shifts.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: Please also refer the comments in the strengths and weaknesses sections:

1. Can the authors compare the al-SAR with other methods under the epoch-based setup to show the effectiveness of the method with larger memory constraint? 

2. Can you consider applying aL-SAR on more diverse datasets (e.g., text, multimodal data) to demonstrate the approach’s generalizability?
soundness: 3
strengths: Pros:

1. The paper introduces a unique approach to online CL by integrating adaptive layer freezing and frequency-based sampling, effectively reducing both computational and memory costs without significant performance trade-offs.

2. The empirical results across multiple datasets comparing to multiple existing SOTA benchmarks show the competitiveness of al-SAR, especially under constrained resource settings.
summary: This paper addresses the challenge of resource constraints in online continual learning (CL) by introducing an efficient method named aL-SAR, which combines adaptive layer freezing and similarity-aware sample retrieval. The adaptive layer freezing component selectively freezes layers to minimize computation based on Fisher Information, while the similarity-aware retrieval component focuses on sampling underused and informative samples from episodic memory to enhance training efficiency. Experiments on CIFAR-10/100, CLEAR-10/100, and ImageNet-1K datasets demonstrate that aL-SAR outperforms several state-of-the-art methods under the same computational and memory budgets.
weaknesses: 1. Though designed to be efficient by only computing class based gradient, the similarity-aware retrieval approach still requires gradient similarity calculations, which could introduce overhead in large-scale applications. 

2. The experimental setups impose strict memory and computational constraints, limiting the contribution primarily to incremental improvements in the efficiency of existing online continual learning approaches.

3. The authors should compare the method with model based like iCaRL.

Meta-review: The paper addresses the challenge of reducing computational overhead and storage requirements in online continual learning (CL). Instead of using training epochs as a measure, it introduces floating-point operations (FLOPs) and total memory size (in bytes) as metrics for computational and memory budgets, respectively. The paper proposes two strategies: similarity-aware retrieval and adaptive layer freezing, which are evaluated through experiments on multiple benchmark datasets.

**Strengths**

- Using FLOPs and total memory size as metrics offers a fair and standardized way to compare different online CL algorithms.
- The proposed adaptive layer freezing and similarity-aware retrieval strategies are effective for computationally efficient online CL, as demonstrated by experimental results.

**Weaknesses**

- Reviewers raised concerns about the additional computational overhead introduced by Fisher information and gradient similarity calculations.
- The impact of the proposed strategies on catastrophic forgetting, a critical aspect of CL, is not clearly established in the paper.

**Overall Assessment**
- The authors’ rebuttal included additional experiments and convincing responses, which adequately addressed the major concerns raised during the review process. The paper provides a meaningful contribution to computationally efficient online CL and has the potential to inspire further research in the field.

Decision: Accept (Spotlight)