Amphibian: A Meta-Learning Framework for Rehearsal-Free, Fast Online Continual Learning
Authors: Gobinda Saha
Kaushik Roy
Venue: Accepted by TMLR ()

Abstract: Online continual learning is challenging as it requires fast adaptation over a stream of data in a non-stationary environment without forgetting the knowledge acquired in the past. To address this challenge, in this paper, we introduce Amphibian - a gradient-based meta-learner that learns to scale the direction of gradient descent to achieve the desired balance between fast learning and continual learning. For this purpose, using only the current batch of data, Amphibian minimizes a meta-objective that encourages alignments of gradients among given data samples along selected basis directions in the gradient space. From this objective, it learns a diagonal scale matrix in each layer that accumulates the history of such gradient alignments. Using these scale matrices Amphibian updates the model online only in the directions having positive cumulative gradient alignments among the data observed so far. With evaluation on standard continual image classification benchmarks, we show that such meta-learned scaled gradient descent in  Amphibian achieves better accuracy in online continual learning than relevant baselines while enabling fast learning with less data and few-shot knowledge transfer to new tasks. We also introduce Amphibian-$\beta$ a unified and principled framework for analyzing and understanding the fast learning and continual learning dynamics. Additionally, with loss landscape visualizations, we show such gradient updates incur minimum loss to the old task enabling fast continual learning in Amphibian.

Published in Transactions on Machine Learning Research (04/2025)
Amphibian: A Meta-Learning Framework for Rehearsal- Free, Fast Online Continual Learning
Gobinda Saha Deportment of Electrical and Computer Engineering
gsaha Brardue edu
Purdue Universily
Kaushik Roy
kaushik Datssku-ppuuuu....m edu
Deportment of Electrical and Computer Engineering Purdue University
Reviewed on OpenReview: https: // openreview. net/ per//formm ia- id=n/AaK0BWbB
Abstract
Online continual learning is challenging as it requires fast adaptation over a stream of data in a non-stationary environment without forgetting the knowledge acquired in the past. To address this challenge, this paper, we introduce Amphibian a gradient-based meta- learner that learns to scale the direction of gradient descent achieve the desired balance between fast learning and continual learning. For this purpose, using only the current batch of data, Amphibian minimizes meta-objective that encourages alignments of gradients among given data samples along selected basis directions in the gradient space. From this objective, it learns a diagonal scale matrix in each layer that accunulates the history of such gradient alignments. Using these scale matrices Amphibian updates the model online only the directions having positive cumulative gradient alignments among the data observed so far. With evaluation on standard continual image classification benchmarks, we show that such meta-learned scaled gradient descent in Amphibian achieves better accuracy in online continual learning than relevant baselines while enabling fast learning with less data and few-shot knowledge transfer to new tasks. We also introduce Amphibian-8 a unified and principled framework for analyzing and understanding the fast. learning and continual learning dynamics. Additionally, with loss landscape visualizations, we show such gradient updates incur minimum loss to the old task enabling fast continual learning Amphibian.
1 Introduction
Autonomous intelligent systems envisioned to operate in non-stationary environments where distribution of online data streams changes over time. In such environments, AT models (usually artificial neural net- works, ANNs) need to acquire knowledge quickly while maintaining the stability of past experiences. This is a challenging scenario as the learning method needs to strike the right balance between learning with- out forgetting and fast learning objectives. However, standard gradient-based training methods for ANNs overwrite the past knowlodge with the information from the new batch of data leading to 'catastrophic forgetting' (28 46). Such forgetting prevents effective knowledge transfer from the past thus also hampering fast learning ability.
To address these challenges, popular line of work in continual learning (CL) B3 In uses memory re- hearsal (3 10
where a subset of past data is stored in a memory bulfer and used with the current batch data to jointly train the model. Such rehearsal- based strategy guides the optimization process such that losses of the past data do not increase, preventing catastrophic forgetting. However, effectiveness of these methods depends on large memory storage which also arises data privacy concerns. In contrast, rehearsal-free methods 20
26 B3
30 3 in continual learning use explicitly designed regularization objectives

---

Published in Transactions on Machine Learning Research (04/2025)
and/or constrained gradient update rules to prevent forgetting. Though these methods are effective in of- fline (multi-epoch) CL. setups, compared to rehearsal-based methods they underperform in online continual learning (OCL) (27) This is primarily due to the added objective or constraints that focus on forgetting mit- igation rather than encouraging fast learning. From fast learning viewpoint, meta-lcarning (4) or 'learning to learn' (22) is an exciting proposition since it optimizes a mela-objective that encourages representation learning ANNs suitable for fast adaptation. Such meta-objective is adapted in [8
g for pre-training
models offline, then deployed for continual learning tasks. In contrast, (32) (6] adapted the meta-objective for fully online continual learning. However, they use memory rehearsal to mitigate forgetting.
In this paper, we propose a meta-learner - Amphibian that learns fast with minimum forgetting during online continual learning. Without any memory rehearsal, Amphibian achieves better balance between learning without forgetting and fast learning with three key components. First, to obtain meta-gradients, it optimizes a meta-objective that on top of minimizing loss on the given batch of samples, encourages their gradient alignments along the selected basis directions representing the gradient space. Sccond, from the same meta-objective, learns diagonal scaling matrices that contain learning/ learning/scaling rates along each gradient basis. In our formulation, each scale value accumulates the history of gradient alignments (along the corresponding basis) among the observed samples over the entire lcarning sequence. Finally, it scales the meta-grann meta-gradients with scaling matrices to update the model along the directions with cumulative positive gradient aligmments among the observed data. Thus, the combination of meta-objective optimization and meta-learned gradient scaling enables Amphibian to learn fast and continually. We evaluate Amphibian in both task- and class- inssemnncrelttlll online continual learning setups (16 40) on long and diverse sequences of image classification tasks (including ImageNet-100) using different network architectures (including ResNet) and achieve better performance in both continual and fast learning metrics compared to the twelve most relevant baselines.
Additionally, we propose a regularized version of Amphibian - Amphibian-8 that offers unified and principled framework for analyzing and understanding the dynamics of fast learning and continual Iearn- ing in popular CL algorithms. Within this framework, by varying a single hyperparameter, 3, we have shown to achieve the functionality of meta-learning- based (this work), gradient-constrre constraint-based [36 37), and regnlarization-baseed (2 50) CL algorithms. We show explicit weight regularization gradient con- straints used in the regularization and conddnet-constrained.e. based methods trade off fast learning ability in favor of mitigating forgetting, hence they underperform in the OCT, setup. Our findings reveal it is hard to design explicit regularization and gradient constraints for achieving the perfect balance between fast learning and continual learning. Hence, Amphibian- like meta-lcarners that automatically learn these constraints from the data would be a promising candidate for this OCL setup.
We summarizee the contributions of this paper as follows:
We introduce Amphibian which minimizes a novel meta-objective and uses meta-learned gradient scaling to enable fast online continual learning without rehearsal.
We provide a detailed derivation of this objective and provide justification for its effectiveness in fast continual learning.
With ovaluation on long sequences of tasks, we show that Amphibian not only learns continually with higher accuracy but also demonstrates the ability of a truly fast learner by learning fast with less data and enabling few-shot knowledge transfer to the new tasks compared to the relevant baselines.
We analyze a regularized version of Amphibian - Amphibian-8 and provide insight that regularized objectives or constraints used by the representative rehearsal-free methods to minimize forgetting restrict the fast learning ability of the model in OCL.
With visualization of loss landscapes of sequential tasks, we show that scaled model update in Amphibian along gradient directions with positive cumulative gradient alignments induces minimum to no increase in loss of the past tasks which enables continual learning.
2

---

Published in Transactions on Machine Learning Research (04/2025)
2 Related Works
Online Continual Learning (OCL). We consider a supervised learning setup where T tasks [TI, (2,,7,,..77 TT)) are learnt by observing their training data [D', D², ..."" sequentially. At any time-step j. the learning model receives batch of data, By or simply B, = { 9%) ) as the set of N; input-label pairs randomly drawn from the current data stream, D'. In online continual learning (27), the model needs to learn from single pass over these data streams with the objective of minimizing the empirical risk on the data from all the
tasks seen so far. The objective [66 43 is given by:
EB EB [O) = EB Iie((,
(1)
Here l(::.) is the loss function to be minimized for task Ti by updating the model parameters
Rehearsal-based Methods. The above risk minimization requires all the data, B1:t-1 from past tasks which may not be accessible to OCL agents the same time. Rehearsal or experience replay (ER) (5 IT)) methods offer solution by storing a limited amount of past data in episodic memory, D-, Such techniques then sample a memory batch, BM- ~ DDM (that approximates B1+-1) for jointly minimizing the objective in Equation ! with the current batch, By. Later works built on this idea where they differ in the way memory is selected and replayed. For instance: GSS (2
selects memory based on gradients, MIR (I) selects memory that incurs maximum change in loss, ASER 40 performs memory selection and retrieval using Shapley value scores. To improve replay, RAR (5T) uses repeated augmented rehcarsal, CLS-ER (3) proposcs dual memory learning and DER (77 uses logit distillation loss. Gradient Episodic Memory (GEM) (26) and Averaged-GEM (A-GEM) ( use memory data to compute gradient constraints for new task so that loss on past tasks does not increase. We refer to (6) [ for a recent literature review online continual learning.
Rehearsal-free Methods. One line of work in this category expands the network (b 48 for contin- ual learning but they are not evaluated for OCL. Other methods under this category minimize the loss (Es, (0; B.))) on the current batch with additional regularization terms and/or constraints on gradient updates. For example, Elastic Weight Consolidation (EWC) (20) and Synaptic Intelligence (SI) (50) add quadratic regularization terms that penalize changes in the important parameters for the past task. While EWC computes parametric importance from the Fisher diagonal matrix after training, SI finds them online during training from the loss sensitivity of the parameters. Gradient projection methods (49 13 36 37) constrain the current gradient, Vel to be orthogonal to the past gradient directions. For instance, Gradient Projection Memory (GPM) (86) optimizes a new task in the orthogonal dircctions to the gradient spaces im- portant for the past lasks, whereas Scaled Gradient Projection (SGP) 37 relaxes the constraint in GPM to allow gradient updates along the old gradient spaces. Natural Continual Learning (NCL) (19) learns in ually by combining gradient projection with regularization. Our proposed Amphibian is also rehearsal-free method. However, unlike these methods it does not need any explicit regularization constraints; rather it mela-kearns from data to scale the gradients for fast continual learning. In the next section, we introduce a meta-learning method - MAML (14 and then discuss relevant meta-learning based continual learning works.
3 Preliminaries
Model-Aguoste Meta-Learning (MAML). MAML (44 is a widely used gradient-based meta-learner that utilizes bi-level (inner-loop and outer-loop) optimization to obtain model parameters, Ag that is amenable to fast adaptation to new tasks. MAMT, trains the model on a set of T tasks simultaneously where each task, TÚ has a dataset, D = {DI; D partitioned for inner and outer loop optimization. One step of inner-loop optimization on task Ti is defined as:
U(05 D.::) = 87 0 VV Lin Di)),
(2)
where U(:.)) is a stochastic gradient descent (SGD) operator, a is learning rate and lm is inner-loop loss function. U can be composed for k such updates as (x DT:) U. o U U(55: Di;) = 0% In the outer-loop optimization, loss for each task is computed at corresponding, 0f on D and expected loss over all the tasks (meta-loss) is minimized (Equation 3) to obtain the paramcters, 8..
min E7,,, [Bout (0% D 0g
(3)
3

---

Published in Transactions on Machine Learning Research (04/2025)
Current data batch: B; (6)) (xf,))), (..-).-----------))
Meta Learning Inner-loop
Meta Leaning Outer-loop
Scale Update
Updated Parameters
Initial Parameters
(4 v6) (x/,))) _.(x (x
Me-11)
B₁
Meta Gradients
A A/+1
0 A
0/ Lin
M
0
A
Imeta (NJ
N
A
Imeta (0/)
Lin
Lin
louc
6
06*1
Model Update
Figure 1: Illustration of continual learning in Amphibian on batch of data. Given the model, og and scale matrix, A3, first we perform k inner-loop gradient update on Og to obtain 0g with the samples of the current batch, S.. In second step, we compute the meta-loss on 0% with the entire batch, B, to obtain meta gradients. In third step we update A and finally the model, og with these gradients.
Meta-learning and Continual Learning. The above meta-loss minimization trains model for fast adaptation, however it does not explicitly encourage continual learning. Thus, (88 proposed online-aware meta- meta-learning where at first, a model is pre-trained offline on set of tasks to learn a better representation for CL, then keeping that representation frozen, the rest of the network is fine-tuned on CL. tasks. The authors in 23 (] used such meta-learning-basee offline pro-training strategy, while allowing varying degrees of adaptation to the model during CL tasks. In contrast, Meta Experience Replay (MER) (32) combines meta- objective of Reptile (30) with memory rehearsal for OCL, whereas La-MAML (I6) minimizes the MAML objective (Equation in online setup where losses on the past tasks are computed on the memory batch, BM- D-M, Unlike these methods, train model from scratch with a meta-objective (Section without rehearsal for fast online continual learning.
Representation of the Gradient Space in Neural Network: In 30 authors show that SGD (gradient) updates lie in subspace spanned by the imput data points. As inside any layer of neural network, the dimension of inputs are smaller than the dimension of the gradients, the gradients thus lie in a low dimensional subspace. Therefore the gradients (or gradient space) in each layer of the ANN can be represented by low dimensional basis vectors. For details see Appendix Section 6. Lct, 0g RCxCC, xkxk represent filters in convolutional (Conv) layer, where C, (Co) is the number of input (output) channels of that layer and & is the kernel size of the filters. Following (36), Og (hence gradient, Vo,e) can be reshaped into a (C; x t x k) x C. dimensional matrix. Thus gradients in a Conv layer can be described by (C; x k x k) dimensional space (instead of Co x C x l x k). Similarly, if Ou E mR=" represents a weight matrix in a cally-connected (FC) layer where m (n) the dimension of outgoing (incoming) hidden units, the gradient space will be n dimensional (instead of m n) in this layer.
4 Continual Learning with Amphibian
Here, we describe the steps (illustrated in Figure I for online continual learning in Amphibian.
Learning Overview: At any time over the learning sequence, Amphibian receives a. batch of data, B, ~ t with N, input-label data pairs from current task, Tr. We aim to update the current model, og using the update rule:
--=-=0//- - (0))
(4)
Here Imett is the gradient obtained from the meta (inner-outer loop) learning process using only the current data, B, In our method, we use the gradient space formulation as described in Section and consider the standard bases (em) of appropriate dimensions the bases of gradient space to represent the gradients inside the neural network. We introduce a scaling matrix, A in the update rule. This is a diagonal matrix; for the convolution layer and FC layer (defined above) it is of size (C; x k x k) x (C; x k x k) and n x n respectively. Each diagonal element, Am, of this matrix is initialized with X and then meta-learned simultancously with Imeta- Over the continual learning trajectory, A accumulates the history of gradient alignment among observed data samples. Then it scales the meta-gradient (Equation 4 accordingly update the model along the direction of positive gradient alignments. Thus A essentially learns the learning
4

---

Published in Transactions on Machine Learning Research (04/2025)
rate of the bases of the gradient space during continual learning. The number of learnable parameters in this matrix is (C; x k x k) for convolution layers and for FC layers. Detailed calculations of the number of learnable gradient scales in our different experiments are given in Appendix Section 5.3. It is to be noted that the gradient (and parameters) in Equation ! are appropriately reshaped before applying the scaled transformation. To learn shared information among lasks during meta-learning, authors (24) introduced layer-wise transformation matrices (T) which (linearly) transform the network activations forward pass and transform (both scale and rotate) the gradients in backward pass. In contrast, we meta-learn shared information for contimual learning across tasks by A matrices that are strictly diagonal matrices and only scale the gradient magnitudes in the backward pass.
Meta-Learning Step-1: At first, on the given batch, B; we perform k inner-loop updates on 0g to obtain 0% as:
k-1 Σ
B.) = - 0g
AAV o l (,
(5)
These inner-loop steps differ in two ways from MAML inner-update step (Equation 2 First, for each inner update, we use one sample (if k = -N)) or subset of samples (if k < N₂) from B, without replacement, whereas MAML uses entire batch (Di, = B,). Second, in our method, each inner gradient is scaled by the A matrix, whereas MAML uses a constant scalar learning rate, a. Though meta-learnable per parametric learning rate (25 16) vector, a and block diagonal proconditioners (11 have been used in such updates, we learn diagonal A which differs in dimensions and interpretation.
Meta-Learning Step-2: In this online learning setup, unlike MAML, Amphibian does not have access to the data from all the tasks seen so far. Moreover, as Amphibian is rehearsal-free learner, we can not store past examples in memory and use them lo approximately minimize the outer-loop MAMI. objective (Equation 3) as in (I6). Instead, in the outer loop of metaaleerning, we compute the meta-loss, lout on current data (B; = Dow)) at 0g and minimize the following objective
min Es, lout (0%: Bi)l min Es, Low (U; %g
(6)
Minimizing this objective with respect to og is approximately equivalent to (see Appendix Section 1 for full derivation):
(ome (cmi((t) Σ X (o))
em T Ilin (0))()))
min EB, Loot (ouu(((;: min og
(7)
30g
30g
where M the dimension of gradient space, Here, lout is computed on entire batch B, while fn is computed on a sample (or subset) from B, First term on the right-hand side of the objective in Equation 7 minimizes the loss current batch of data, B:. The second term encourages positive alignment. (inner product) of gradients computed samples of B, along selective gradient basis directions (em) depending on scale Am- For instance, for positive Am: inner product of data gradients along em is maximized whereas for zero negative Am; such gradient alignments are not encouraged. Gradient of this objective is given by Imeta (0)) which is used for model update in Equation
Scale Update Step: Next, gradient of the meta-objective (in Equation 6) with respect to each scale, X in A can be simply expressed (using first-order approximation (14 as follows:
Imeta (Nin)) = (l)
dlm((%)) 00
em mm
Lone (0)) em T
i
Σ (o.))
(8)
k==0
W=0
Derivation is given in Appendix Section 2. Equation denotes that if outer-loop gradient gout = Coue (0 and accumulated inner-loop gradient Gon Ex Cy
( (0%) has positive inner product (aligned) along em: then gmneta (N))) will be negative, whereas if the inner product is zero (negative) then gmntt (N%,) will be zero
5

---

Published in Transactions on Machine Learning Research (04/2025)
e2
A2
A gout
Jin
k'
scale
gin
Bout
interference
A
Figure 2: Conceptual illustration. Along e2 (e1) Jout and Isn has alignment (interference), hence X2 (A1) will increase (decrcase). Considering, Imeta (8%) = alignment history.
Pout: Imeta is scaled accordingly in model update to reflect
(positive). The scale update rule: N+ Min
T1ymnta (N/)) can be expressed as:
1-1 (0%)) = -o + 17 Σ Loue (0a) Cm c
W+1 = =//n + nrooe (6%) Cm C
Σ en (fk))))
(9)
where 1) is the learning rate for the scales. This update rule provides two valuable insights. First, the value of a scale will increase (decrease) if, along the corresponding basis, em direction outer- and inner-loop gradient trajectories have positive (negative) inner product or alignment (interference). Figure provides conceptual illustration this point. Here 6₁ and 62 represent the gradient basis directions and X1 and X are associatod scales. Since along e2; Jout and gin has positive inner product, A2 will increase. Whereas Åι will decrease, along é1 components of 9out and Iin has negative inner product. Second, over the entire continual learning sequence up to time j. Xb accumulates the history of such gradient alignments or interferences. Now if we do not use accumulation and update the scales as W = Ao M1meta (M/) with current Imeta (X,)) and use these scales for model update (Fquation 4), we would get fast learning on the current data but model will forget past data. In contrast, by using cumulatively updated (Equation 9 in Equation 4 we ensure that the gradient step on each incoming data is accelerated along the direction of positive cumulative alignment, whereas blocked along the direction of zero negative alignments thus minimizing catastrophic forgetting. Therefore, this online accumulation of gradient alignments in scale matrices over the learning sequence enables Amphibian to learn continually with minimum forgetting without any data rehearsal. An analysis of the evolution of A during training is given in Appendix Section 5.4 in support of this point.
Finally, with (meet(ff) and the updated scale A++1 we perform model update on the current batch as Equation 4 As the scales, Am can take both positive and negative values, to prevent gradient ascent (both in Equation 4 and inner loop Equation 5 we only use their positive part using (Am)+ = 20,,0 function, where 1.20 of this function using the straight-Chrough estimator (5) 45 With the updated model 0f+1 and scale matrix Ai+1, we (continually) learn the next batch of data. The pseudocode of the algorithm is provided in Algorithm 1 with more details provided in Appendix Scction 3.
Model Update Step:
1). In the meta-learning steps, we ensure diffcrentiability
5 Experimental Setup
Datasets and Models. We evaluate Amphibian and the baselines in online continual learning (OCL) (
setups where models learn from the single pass over the data stream. We use 5 standard image classification
benchmarks in continual learning: 20 tasks split CIFAR-100 (16), 40 tasks split Tiny Imygnntenet (II). 25
lasks split 5-Datasets (BB). 20 lasks split Net (7) and 10 tasks split minilmageNeet (40)
Similar to (16) IT]. use 5-layer network for CIFAR-100 and 5-Datasets, and 6-layer network for Tiny-
Imagenet. For Tet and minilmageNet experiments we use ResNet-18 (26) model. Details on the dataset statistics/splits, and network architectures are provided in the Appendix Section 4.1 and 4.2 respectively.
Basclines and Training. We compare Amphibian with rehearsal-free methods: EWC 20 and SI (50)
which use parametric regularization; GPM (36) and SGP (37) which use gradient projection; and NCL 19 which uses both gradient projection and regularization. Although comparisons with rehearsal-free and

---

Published in Transactions on Machine Learning Research (04/2025)
Algorithm Amphibian Algorithm for Online Continual Learning
1: Imputs: 8: noural network parameters (weights), lin: inner-objective, lout: outer (meta) objective, Ao; initial values for all the scales, op scale learning rale, T: number of lasks.
j+ 0, 0g +-
Initialize For weights/filters in each layer initialize diagonal matrix A with X
A 4 initialize (19)
4: for T. E 1,,, T do
5: for batch, B; ~ D do
DD is data stream of current task
6: k + size)BBB)
7: for k' = to - do
8:
# = Og \/ 0% en (f: B.(k)) end for
Inner-loop updates
10:
N Nn 11 Nin Loutt 00z: Bi)
Update scales in A
11:
0 1- 0g N-1V og agloau (0 Bi)
Update model parameters
12:
j j+3 j +1
13: end for
14: end for
beaee methods are not always fair (especially with large data memory), compare with : ER (TO). having moderate data memory (100 to 400 samples). which uses meta-lcarning and memory rehearsal for OCL. Following baselines, we do not use any oflline pre-trained models, rather we train models from scratch. In Amphibian, scale learning rate (1) and initial scale value (A%, hyperparameters were set with grid search (as in (16) with held out validation data from training sets. Similarly, all the hyperparameters of the basc- lines were tuned. Details of training setup, implementations and list of all the hyperparameters considered in the baselines and our method are given in Appendix Section 4.
GEM (26), A-GEM (9), DER++ (7 and CLS-ER K3
We also compare with MER 32 and La-MAML 16
Evaluation Metrics. We measure OCL performance with two metrics: ACC measures average test classification accuracy of all tasks and BWT (backward transfer) measures influence of new learning on the past knowledge with negative BWT denotes forgetting. They are defined as:
T Σ RTS: BWT T ACC T
Σ RT, R
(10)
Here, T is the total number of tasks and Ri, is the accuracy of the model on 7th task after learning the thh task. Higher ACC in ouline setup signifies fast continual learning ability, however to gain better insight on the fast learning ability we introduce two additional metrics task learning efficiency (TLE) and few-shot forward transfer (FWT TLE (15) is defined as the size of D, required to achieve certain (7%) classification accuracy on that task, t. This metric implies if less data (small TLE) is required to achieve a certain performance level, then the model can transfer knowledge faster, hence is a fast learner. For comparisons among the baselines, we set y as 90% of the final Amphibian accuracies on each task, t. Additionally, in meta-learning (44 30), fast learning capability is measured with N-way K-shot adaptation accuracy where meta-learned model is trained
NK examples with few gradient steps and then tested. Thus truly fast learner should also perform well under this setup. To evaluate this, after learning each OCL task, & we sample K examples from each of the N classes from the next task and fully adapt the model for n, steps. Then record the test accuracy on this new task as FWT,s the few-shot forward transfer capacity of the model learned after task For all the tasks we measure and compare this capacity as FWTFS -I FWT
6 Results and Analyses
6.1 Continual Learning and Fast Learning Performance Comparisons
Accuracy and Forgetting. First, we evaluate and compare the performance in task-ineremental OCL setups where each task has a separate classifier and task identity is used during inference. In Table [ we provide comparisons of ACC and BWT among various methods within this setup across 3 different
7

---

Published in Transactions on Machine Learning Research (04/2025)
Table 1: Performance (mean + std from 3 runs with random seeds) comparisons in online continual learn- ing. (*) indicates results for CIFAR-100 and Tinylmagenct are taken from (16)
CIFAR-100
TinyImagenet
5-Datasets
Rehearsal
Methods
ACC (%)
BWT (%)
ACC (%)
BWT (%)
ACC (%)
BWT (%)
ER*
47.8 0.73
12.4 0.83
39.3 + 0.38
14.3 0.89
83.4 0.69
9.43
1.24
GEM
48.2 1.10
13.77 = 0.70
40.5 1 0.79
13.5 + 0.65
86.9 1.09
7.43 0.61
A-GEM*
46.9 0.31
-1344 = 144
38,9 1 0.47
-13.6 1 1.73
82.4 = 1.18
9.66 = 0.82
MER
13.4 1.44
38.9 1 0.47
13.66
88.5
0.33
DER+
53.4 1.75
8.16 1.12
44.1 1: 0.29
14.5 1: 0.49
86.7 0.86
4.3 + 1.09
CLS-ER
60.5 0.45
0.90 + 0.50
51.8 + 1.01
15.9
0.71
89.1 1.29
7.13 + 0.09
La-MAML*
61.1 1.44
9.00 = 0.20
52.5 + 1.35
1.22 3.70
89.0 1.45
5.97 2.00
EWC
50.4 1.88
1.53 = 0.98
43.6 + 0.83
0.83
1.17
80.1 1.76
8.06 1.22
SI
51. 0.69
0.73 0.24
42.3 1 3.43
1.60
0.14
80.9 1.71
5.96
2.45
No
GPM
59.1 1.26
0.00
0.12
48.3 1 2.69
0.30 1 1.00
84.3 2.57
2.06
0.94
SGP
61.3 1.25
0.00 = 0.13
52.7 + 0.26
0.00
0.44
86.1 3.52
5.56
3.20
NCL
0.00
0.18
0.91
0.83
85.5
0.00
0.05
No
Amphibian
65.0
0.96
1.30
0.25
54.8
0.72
1.24
0.96
(a)
(b)
(c)
(d)
60
Amphiblan
100
Amphibian
Average TLE (%)
Average TLE (%)
TUE (%))
80
TLE (%6
40
40
20
40
40
20
Task ID
CIFAR-100
Task ID
TinylmageNet
EWC
GPM
SGP
NCL
Amphibian
Figure 3: (a) Task learning efficiency (TLF) of Amphibian. (b) Average TLE (lower the better) of all the methods for CLFAR-100 tasks. (c) TLE of Amphibian and (d) Average TLE all the methods for Tinylmagenet.
datasets. Among the rehearsal-based methods, meta-learner La- MAML achieves better accuracy. Even without memory rehcarsal, Amphibian outperforms La-MAML (by up to ~8.5% in ACC) with less forgetting. Since the rehearsal-free methods are under-explored for OCL and Amphibian is a rehearsal-free method, the following analyses and discussions we focus primarily on rehearsal-free methods. EWC and ST achieve similar performance, however, compared to Amphibian they severely underperform (by up to 14% in ACC). regularization used in these methods to ensure stability
This shows that the parametric importance-basee of knowledge (as indicated by small forgetting in Table 1 is not favorable for fast. learning required in OCL. In contrast, gradient projection methods such as GPM and SGP achieve up to ~00 ACC gain over EWC and SI. However, Amphibian outperforms these methods by up to ~4% in ACC with marginally higher forgetting. This shows without any explicit constraints of orthogonal gradient projections (as in GPM and SGP), Amphibian properly learned to scale the gradient updates from the online data to perform better in OCL. NCL performance lies in between gradient projection and regularization--asee methods and is outperformed by Amphibian. Moreover, in challenging 5-Dataset tasks where data arrives from dissimilar domains over time, Amphibian obtains the best performance over all the methods. Next, in Table 2 compare Amphibian's performance with the basclines on 20 split ImageNet-100 tasks using the ResNet-18 model. Here, Amphibian archives ~ 2.4% better accuracy, demonstrating its scalability to larger dalasets and complex networks. We use a multi-glance OCL setup for CIFAR-100 and linylnetenet experiments to ensure direct comparability with the results reported in (16). However, we used single glance OCL sctup for 5-Datasets and ImageNet- 100 experiments to demonstrate Amphibian's effectiveness in this configuration well.

---

Published in Transactions on Machine Learning Research (04/2025)
(a]
(b)
(c)
Average Accuracy (%)
50
CLS ER Amphibian
Memory (Normalized)
20
FW/T (%)
GPM
102
20
La-MAML
101
10
100
6
7 B 10
CIFAR-100
TinylmageNet
CIFAR-100
TinylmageNet
Task Number
GPM
La-MANL
Ampribian
Figure 4: (a) Average accuracy for minilmageNet tasks in online class-incremental learning.(b) Few-shot forward transfer, FST (higher the better). (c) Training time memory overhead comparisons.
Finally, we evaluate our method a class-incremental OCT sctup. Following (40), we train ResNct-18 model from scratch 10 minilmageNel tasks and during inference, task identity is not used. In Figure 4a), we compare the average accuracy of the model trained with different rehearsal-based and free meth- ods. After 10 tasks, our method achieves better accuracy than the best. rehearsal-baset method CL.-ER. However, we ac- knowledge that Amphibian's performance in class- incremental learning (CIL) is not as strong as in the task-incremental Iearn- ing setup. Rehearsal-free CIL is very challenging, but we believe our proposed framework will serve a solid foundation for future research in this area. All the following analyses are performed in task-incremental setups.
Table 2: Performance comparisons for 20 IamgeNet-106 tasks on RosNet--1.
ImageNet-100
Methods
ACC (%)
BWT(%)
GPM La-MAML
45.5 + 1.33
0.00 + 0.23
51.7 + 1.15
5.60 +: 0.80
Amphibian
54.1 + 0.76
0.34 + 0.20
Task Learning Efficiency (TLE). As discussed in Section 5 fast online continual learner should achieve high performance with less amount of data (smaller TLE per task). In Figure 3(a) a) we show TLE of Amphibian for each CIFAR-100 task. Averaging over all the tasks we obtain an average TLE of 21% for Amphibian. This means, on average, a new task in Amphibian can be learned to 90% of its final achievable accuracy by learning 21% of the data from that task. In Figure 3
3()) TLE of Amphibian for each TinyImageNet task is shown which gives an Average TLE of 35%. We compare the Average TLE of all the rehearsal-free methods in Figure 3 ()) and (d). In both dalasets, Amphibian oulperforms all the other methods, indicating that Amphibian can obtain high accuracy level on given tasks very quickly by observing fewer examples from the data stream, hence it is a fast learnor.
Few-shot Forward Transfer (FWTFs). ) In Figure 4 4b) we compare FWT Es of different algorithms for CIFAR-100 and Tinylmagenet datasets, where higher FWTFs means better few-shot (rapid) learner. For each PWTPs, we use 5-way 5-shot training data and adapt the network with n, 10 steps. Compared to the rehearsal-free methods, Amphibian achieves up to 5% better FWT which demonstrates the fast learning ability developed in the model from online continual learning in Amphibian. Here we show the performance of (meta-learned) La-MAML, which also achieves better performance than other rehearsal- free baselines but outperformed by Amphibian. This shows meta-learning plays a key role in building fast learning capability in Amphibian and MT-M
Memory overhead and Training Times. In Figure 4 (c) we compare the memory overhead during training in each method for either storing the old model, important parameters and/or past data. The numbers are normalized by the memory overhead of Amphibian. For both datasets, all the baseline methods use orders of magnitude more memory than Amphibian. For instance, where Amphibian only requires up to ~ 0.5% extra memory compared to network size for gradient scale (A) storage (scale nunbers given in Appendix Section 5.3), La-MAML requires 100% extra memory for per parametric learning rate storage and up to 179% extra memory for replay buffer. Among all the methods NCL has the highest memory overhead for model and projection matrix storage. Thus Amphibian most menory-effeiiett learner for OCL. Training time comparisons are provided in Appendix Section 5.1.
9

---

Published in Transactions on Machine Learning Research (04/2025)
Table 3: Comparison of Amphibian with candidate rehearsal free meta-learning methods for OCL.
CIFAR-100
Methods
ACC (%)
BWT(%)
FWTes(%)
ACC (%)
BWT(%)
FWT
Amphibian
65.0 + 0.96
1.30 0.25
47.6
1.97
0.60
0..72 = 0.22
10.6 0.87
La-MAML (No Rehransal)
56.1 4: 0.87
12.8
0.17
46.7
1.89
49.2 d: 0.55
7.23 à: 0.16
39.5 à: 0.75
Online Meta-SGD
51.3 + 2.51
15.7
2.50
45.9
+ 1444
15.1 # 1.31
10.7 = 0.57
37.7 = 0.44
Online MAMI.
49.0 + 1.81
18.0 + 2.00
42.2
1.48
40.4 + 0.57
8.10 + 0.74
31.8 + 0.45
(b)
(c)
(d)
80
CIFAR-100
Tinylmagenet
Po
ACC (%)
BWT(%)
ACC (%)
60
Bg B
-2
B.
-4
40
1e-?
-6
P1 lee
40
20
1e-3
1e" 2
1
1e-3
1e-2
1e
44
46
48
39 FWTFs (%)
B
FWTFs (%)
CIFAR-100
Tinylmagenet
5-Datasets
Amphiblan
EWC
GPM
Figure Variations of (a) ACC, (b) BWT, with 3 in Amphibian-3. ACC vs. FWTFs comparisons for various Amphibian-3 with rehcarsal-free froe basclines for (c) CIFAR-100 and (d) TinyImagenct.
6.2 Amphibian vs. Online Rehearsal-free Meta-Learning
To our knowledge, there are no rehearsal-free meta-learners for OCL, so we adapt popular meta-learning approaches for OCL and compare them with Amphibian in Table 3 First method is La-MAML(No Rehearsal), where we remove the memory buffer from La-MAML and compute the meta-loss with only current data (as Amphibian). The notable difference between this method and the Amphibian is that it learns learning rates for each parameter and uses that in inner- and outer-loop gradient updates, whereas Amphibian learns a diagonal scale matrix at each layer and scale the gradient directions accordingly. From T we find that La- La-MAML.(No Rehearsal) vastly under-performs Amphibian, particularly it suffers from large forgetting (up to 11.5% more than Amphibian). This shows the novel meta-objective optimization and gradient scaling with the learned scale matrix (A) enable Amphibian replace the meta-learned learning rale in outer-loop update from La-
learn continually without
rehearsal. In second method, MAML(No Rehearsal) with a constant learning rate, whereas inner-loop still uses learnable learning rates. This converts the method to online Meta-SGD (2 Results show that ACC drops further, forgetting increases and fast learning capability reduces. Finally, we use constant inner- and outer-loop learning rates in online Meta-SGD, which converts the method to online MAML (1.. With no learnable scale matrix or learning rate to encode the history of past data, this method performs the worst in OCL setup. In summary,
these analyses clearly show the functional difference between our method and La-MAML and MAML, and highlight the importance of meta-learned gradient scaling in Amphibian for fast OCL performance.
6.3 Analysis of Amphibian: Amphibian-8
Here, we introduce a regularized version of Amphibian Amphibian-8 to understand the relationship be- tween fast learning and forgetting. Amphibian-8 uses the same online continual learning steps as Amphibian except it performs model update using the following rule:
t = 0g - -A51 Imeta (0)) -30) - 0.) = =
_ 21--38-1 ===0
3 1'''' +1+1 Imeta ( (0,')
(11)
So far, used was used as the time index over the entire learning sequence (spreading across tasks). Here, we denote j as jth time step task T,. Thus, at the start of task T. 86 would denote optimum parameter learned till task 7-1- Here, B E [0, 1]. For = 0 it reduces to Amphibian update whereas for B = the
10

---

Published in Transactions on Machine Learning Research (04/2025)
model parameter stays at initial point gg. Thus increasing 3 from 0 regularize the Amphibian update to stay near the solution of the past lasks. This is a useful concept. in continual learning staying close to the old solution point provides a degree of protection against catastrophic forgetting. In Figure 5 aa) and
(b) we show how ACC and BWT vary when & is varied (from to 0.1). For all datasets, increasing B reduces lorgetting in Amphibian (Figure 5 [b)) especially for B > le-2 there is no forgelting. However, with increasing 8, ACC degrades (Figure a.)). These results indicate that explicit regularization for forgetting reduction restricts the fast continual learning capacity of Amphibian. In Figure ((c) and (d) we plot ACC FWTFs for Amphibian-3 with rehearsal-free baselines for CIFAR-100 and Tinylmagenet. These plots show as we increase B3, both continual and fast learning performance of Amphibian becomes similar to the baselines. Such functional similarities provide valuable insight that explicit regularization or constraints used in these rehearsal-free methods primarily focus on forgetting miligation at the expense of fast learning, hence they underperform in OCL setup. These analyses call for a rethinking of regularization or constraint design in rehcarsal-free methods and provide motivation for exploring Amphibian-like learner that learns the required constraints for optinal balance between fast and continual learning.
6.4 Understating Continual Learning Dynamics in Amphibian
Continual learning works on the principle that learn- ing a new task should not (or minimally) increase the loss of the old tasks. Amphibian does not explicitly minimize any such objectives, instead, it continually updates models in the direction of positive cumula- live gradient alignments among the observed data. To understand how Amphibian enables continual learn- ing we use loss landscape visualizations
(a) Task test
(b) Task 5 test loss
2
e
1
1.0
1.5
915
1.3
0.5
1.1
0.0
0.9
0
(
05
(
0.7
(43 29). In Figure a(a) we plot the loss contour of task from CIFAR- 2D plane. Here O' indicates the net- work model after learning task t. Here, the black line indicates the learning trajectory, with each cross point representing the projection of learned models on the plane along the trajectory. In Amphibian when we sequentially learn from task 1 to task 20, along the learning trajectory (0) + 010 + 0 in Figure 6 a()) loss of task 1 only increase minimally from the initial point (0)). A similar trend can be seen for task 5 (in Figure b)), where along the learning trajectory (A f15 - 020) loss increases in task 5 minimal. Similar pattern is also found for Tinylmagenet tasks (Appendix Scction 5.2). Thus, in Amphibian, model updates along the directions with positive cunulative gradient alignments prevent significant increase in the loss of past data enabling continual learning with minimum forgetting.
0.5
-055
0.0
0.5
1.0
-0.5
0.0
0.5
1.0
Figure 6: Dynamics of continual learning in Am- phibian. Loss contour of (a) Task and (b)
Task 5 plotted on 2D planes defined by parameters (0), 010 ,000) and (05, ,055 020) respectively.
7 Conclusions
In this paper, we introduce rehearsal-free meta-learner- Amphibian that in a fully online manner learns to scale the gradient updates to enable fast online continual learning. To this end, Amphibian optimizes a novel meta-objective and learns scale matrices that accumulate the history of gradient alignments among the data samples observed over the learning trajectory. Using these scale matrices it updates the model in the direction of positive cumulative gradient alignments. On various continual image classification tasks, we show that such meta-learned scaled gradient update in Amphibian enables memory-effeieet, data-efficient, and fast online contimual learning. In conclusion, we believe Amphibian offers a unified framework for exploring meta-lcarning and continual learning, making it a valuable tool for dissocting inherent tradc-offs and ultimately facilitating the development of improved algorithms that strike desired balance between fast and continual learning.
11

---

Published in Transactions on Machine Learning Research (04/2025)
Acknowledgments
This work was supported by the Center for the Co-Design of Cognitive Systems (COCOSYS), DARPA sponsored JUMP center of Semiconductor Research Corporation (SRC), National Science Foundation and United States Department Energy.
References
[1] Aljundi, R., Belilovsky, E., Tuytelaars, T., Charlin, L., Caccia, M., Lin, M., Page-Caccia, L.:: Online continual learning with maximal interfered retrieval. In: Wallach, H., Larochelle, H., Beygelzimer, A., d' l'Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 32 (2019)
[2] Aljundi, R., Lin, M., Goujaud, B., Bengio, Y.: Gradient based sample selection for online continual learning. In: Wallach, H., Larochelle, H., Beygelzimer, A., d' d'Alché-Buce Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 32. Curran Associates, Inc. (2019)
[3] Arani, E., Sarfraz, F., Zonooz, B.: Learning fast, learning slow: A general continual learning method based on complementary learning system. In: International Conference on Learning Representations (2022), htttp::/openreevee. wtt.:er.......oom.mooddddoooorrr
[4] Beaulicu, S., Frati, L., Miconi, T., Lehman, J., Stanley, K.O., Clune, J., Cheney, N.: Learning to continually learn. arXiv preprint arXiv:2002.09571 (2020)
[5] Bengio, Y., Léonard, N., Courville, A.: Estimating propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 (2013)
[6] Bidaki, S.A., Mohammadkhah, A., Rezace, K., Hassani, F., Eskandari, S., Salahi, M., Ghassemi, M.M.: Online contimual learning: A systematic literature review of approaches, challenges, and benchmarks (2025), https: www./rg////baaeeee 04897
[7] Buzzega, P., Boschini, M., Porrello, A., Abati, D., Calderara, S.: Dark experience for general continual learning: strong, simple baseline. In: Advances in Neural Information Processing Systemss. vol. 33, pp. 15920-15930, Curran Associates, Inc. (2020)
[8] Caccia, M., Rodriguez, P., Ostapenko, O., Normandin, F., Lin, M., Page-Caccia, L., Laradji, I.H., Rish, I., Lacoste, A., Vázquez, D., al.: Online fast adaptation and knowledge accumulation (osaka): new approach to continual learning. Advances in Neural Information Processing Systems 33, 16532 16545 (2020)
[9] Chaudhry, A., Ranzato, M., Rohrbach, M., Elhosciny, M.: Efficient lifelong learning with A-GEM. In: International Conference on Learning Representations (2019)
[10] Chaudhry, A., Rohrbach, M., Elhosciny, M., Ajanthan, T., Dokania, P.K., Torr, P.H.S., Ranzato, M.: Continual learning with tiny episodic memories. ArXiv abs/1902.10486 (2019)
[11] Deng, D., Chen, G., Hao, J., Wang, Q., Heng, P.A.: Flattening sharpness for dynamic gradient pro- jection memory benefits continual learning, Advances in Neural Information Processing Systems 34, 18710-18721 (2021)
[12] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248-255. Icce (2009)
[13] Farajtabar, M., Azizan, N., Mott, A., Li, A.: Orthogonal gradient descent for contimual learning. In:
International Conference on Artificial Intelligence and Statistics. pp. 3762-3773. PMLR (2020)
144 Finn, C., Abbeel, P., Levine, S.: agdel-agnostic meta-learning for fast adaptation of deep networks. In: International conference machine learning. pp. 1126-118 1126-1135, 1135. PMLR (2017)
12

---

Published in Transactions on Machine Learning Research (04/2025)
[15] Finn, C., Rajeswaran, A., Kakade, S., Levine, S.: Online meta-learning. In: International Conference on Machine Learning, pp. 1920-1930. PMLR (2019)
[16] Gupla, G., Yadav, K., Paull, L.: Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems 33, 11588-115111 (2020)
[17] Hadsell, R., Rao, D., Rusu, A.A., Pascanu, R.: Embracing change: Continual learning in deep neural (2020)
networks. Trends in cognitive sciences 24(12),
[18] Javed, K., White, M.: Meta-learning representations for continual learning. Advances in neural infor- mation processing systems 32 (2019)
[99 Kao, T.C., Jensen, K., van de Ven, G., Bernacchia, A., Hennequin, G.: Natural continual learning: success a journey, not (just) a destination. Advancos in Neural Information Processing Systems 34, 28067 28079 (2021)
[20] Kirkpatrick, J., Pascanu, R., Rabinowitz, N.C., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska- Barkinkk,,, A., Hassabis, D., Clopath, C., Kumaran, D., Hadsoll, R.: Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114, 3521 3526 (2017)
[21]
Krizhevsky, A.: Learning multiple layers of features from tiny images. Tech, rep. (2009)
[22] Le, Y., Yang, X.S.: Tiny imagenet visual recognition challenge (2015)
[23] Lee, E., Huang, C.H., Lee, C.Y.: Few-shot and continual learning with attentive independent mecha- nisms. In: Proceedings of the IEEE/CVP International Conference on Computer Vision. pp. 9455 9464 (2021)
[4]] Lee, Y., Choi, S.: Gradient-basee based meta-learning with learned layerwise metric and subspace. In: Inter- national conference on machine learning. pp. 2927-2992 2936. PMLR (2018)
[25] Li, Z., Zhou, F., Chen, F., Li, H.: Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.0988 09835 (2017)
[26] Lopez-Paz, D.,, Ranzato, M.A.: Gradient episodic memory for continual learning. In: Advances in Neural Information Processing Systems. vol. 30 (2017)
277] Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., Sanner, S.: Online continual learning in image classifi- cation: An empirical survey, Neurocomputing 469, 28 51 (2022)
(88) Mccloskey, M., Cohen, N.J.: Catastrophic interference in connectionist. networks: The sequential learn- ing problem. The Psychology of Learning and Motivation 24, 104-169 (1989)
[29] Mirzadeh, S.I., Farajtabar, M., Gorur, D., Pascanu, R., Ghasemzadeh, H.: Linear mode connectivity in multitask and continual learning, arXiv preprint arXiv:2010.04495 (2020)
300 Nichol, A., Achiam, J., Schulman, J.: On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02m8 02999 (2018)
[11] Park, E., Oliva, J.B.: Meta-curvature, Advances in Neural Information Processing Systems 32 (2019)
[32] Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., Tesauro, G.: Learning to learn without
forgetting by maximizing transfer and minimizing interference. In: International Conference on Learning Representations (2019)
[33] Ring, M.B.: Child: A first step towards continual learning, In: Learning to Learn (1998)
[34]
Robins, A.V.: Catastrophic forgetting, rehearsal and pseudorehearsal. Connect. Sci. 7, 123 146 (1995)
335] Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukeuoglu, K., Pascanu,
R., Hadsell, R.: Progressive neural networks. ArXiv abs/1606.04671 (2016)
13

---

Published in Transactions on Machine Learning Research (04/2025)
[36] Saha, G., Garg, L,, Roy, K.: Gradient projection memory for continual learning. In: International
Conference on Learning Representations (2021),
[37] Saha, G., Roy, K.: Continual learning with scaled gradient projection. Proceedings of the AAAI Confer- ence Artificial Intelligence 37) 9677-9687 (Jun 2023). https://doi.orr/100 http:///000.000//.a/aaaa.0a. v3718.26157
(88] Saha, G., Roy, K.: Saliency guided experience packing for replay in continual learning. In: Proceedings of the LEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 5273 5283 (Jamuary 2023)
[39] Serrå, J., Surís, D., Miron, M., Karatzoglou, A.: Overcoming catastrophic forgetting with hard attention to the task. In: Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 4548 4557. PMLR (10-15 Jul 2018)
[40] Shim, D., Mai, 7., Jeong, J., Sanner, S., Kim, H., Jang, J.: Online class-incrementee continual learning with adversarial shapley value. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 9630 9638 (2021)
41] Son, J., Lec, S., Kim, G.: When meta-lcarning meets online and continual learning: A sur- vey. IEEE Transactions on Pattern Analysis and Machine Intelligence 47(1), 413 432 (2025). https://doi.or /AA.100/TAAML2222446366633
[2] Thrun, S., Pratt, L.: Learning to learn. Springer Science & Business Media (2012)
[33] Verwimp, E., De Lange, M., Tuytelaars, T.: Rehearsal revealed: The limits and merits of revisiting samples in continual learning. In: Proceedings of the LEEE/CVF International Conference on Computer Vision. pp. 9385 9394 (2021)
[44] Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. Advances in neural information processing systems 29 (2016)
[45] Von Oswald, J., Zhao, D.,, Kobayashi, S., Schug, S., Caccia, M., Zucchet, N., Sacramento, J.: Learning where to learn: Gradient sparsity in meta and continual learning. Advances in Neural Information Processing Systems 34, 2250-5266 (2021)
[66] Wickramasinghe, B., Saha, G., Roy, K.: Continual learning: A review of techniques, chal- lenges and future directions. IEFE Transactions Artificial Intelligence pp. 1-21 (2023). http///....rg111110//11111
[47] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014-3023 (2021)
[88] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. In: 6th International Conference on Learning Representations (2018)
49 Zeng, G., Chen, Y., Cui, B., Yu, S.: Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence 18), 364 372 (2019)
50] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 3987 3995. PMLR (06 11 Aug 2017)
51] Zhang, Y., Pfahringer, B., Frank, E., Bifet, A., Lim, N.J.S., Jia, Y.: simple but strong baseline for online continual learning: Repeated augmented rehearsal. arXiv preprint arXiv:2209, 139177 (2022)
14

---

Published in Transactions on Machine Learning Research (04/2025)
Appendix
Derivation of Amphibian meta-objective and meta-gradients are provided in Section | and 2 respectively, A detailed explanation of the Amphibian algorithm is given in Section 3 Experimental details including dataset statistics, network architectures, a list of hyperparameters along with implementation details are provided in Section Additional results and analyses are provided in Section 5
1 Derivation of Amphibian Meta-Objective
In this section, we will show that when we optimize the following meta-objective in Amphibian:
min Es, Low (0%: Bx) min Es, [ooue (0) :.) (c)]
(A.1)
%
where each of the h inder-update is taken using sample (or subset of samples) from current batch, B, from task Ti and the meta-loss, lot is computed on the entire current batch data, Bi it is equivalent to minimizing the following objective:
M Σ
(l)))(h)) 20g
Hlen (0%)
min %
(one(0))
X
em e
(A.2)
00g
m=1
For that, let us define,
Hl (0%)
(gradient of meta-loss at 0%)
(A.3)
9k
00%
Hlout (0))
0k
(gradient of meta-loss at 0g)
(A.4)
009
Ol, (0%)
9k'
(gradient of inner-loss at 6g where k' < k)
(A.5)
201
Dl (0%)
9N
(gradient of inner-loss 0g where k' k)
(A.6)
a6g
0/ = 0%. A gk.
(sequence of parameter vectors)
(A.7)
H C" (6%)
(Hessian of meta-loss at 0g
(A.8)
Hy the (0)) (Hessian of inner-loss at 0
(A.9)
First, let's write the gradient of meta-loss (outer-loop loss) at 0% from Equation A.3 as (using Taylor's expansion 30)):
9k - Cone (0%) (8)) Ch (8) 0%) + +0102 - -611=)
(' implies derivative w.r.t argument)
Th + (0) 0g) 0(110) 08113) (using definition of Ōk: Hx)
k-1
k-1
Gz H
A# gw + O(A3) (using 0% -/8 =
A gv, from Equation 5)
(A.10)
==0
==
N gw + O(A²) (using gk TN O(A))
=D
k-1
H
Th + O(A2) (let scaled update, z = A² gg
k= 0

---

Published in Transactions on Machine Learning Research (04/2025)
Now, let's derive the meta-gradient (or MAML gradient (4)
for parameters by. denoted as gmeta (8)
OL oou (02)
Olour (0%)
OU
Imeta (0))
06%
00%
06/
OU
(0)))
yh
00%
(repeatedly applying chain rule and using, 0f = U(00-1))
(A.11)
II
10% - ~A/wx)
gk
00%
k/
k-1
0 (0% skd)
005
gk
k-1 1
(T H (where, H is Hessian of scaled gradient, g at 0%
k/=00
Using Taylor's theorem and dropping higher order (O(A²) terms B0 we can write HD HD and then using 9k from Equation A.10 in Equation A.1I we get:
k-1 (I H
Imeta (0/)
I1
9k
HR
gk
+ +(AA)
1-1
k-1
)
H
Di
+ 0(13)
(A.12)
=0=
k==0
H
H Σ
Φh + 0(A3)
9k
k'
km=0
Now, using & = 1 in Equation A.12
derive the equivalent objective in Equation A.2 For higher k, the form of objective becomes complicated but has a similar set of terms. Thus putting k in Equation A.12
al (0%)
Imeta (0%) 91
H6 91
H196 + 0(A3)
0/
0
a
= 91
(g1
g6')
(using
(g1 git = H6' g1 + +(g
00g
20g
aloue
a
A Dlin (0%))
(A.13)
a08
a0g
00g
20%
(expressing terms as derivatives)
(cmee(())
Σ t Olou(())
T Hhin (0))
C. e
2eg
aeg
ae
which is precisely the gradient of the Amphibian meta-objective in Equation A.2
2

---

Published in Transactions on Machine Learning Research (04/2025)
2 Derivation of Meta-Gradients
In this section we first derive the mota-gradicnts of scales Xn which is defined as:
(l)
Hl alou
00%
Imata (Mm))
0Nm
00%
2N
Olou
(U(O)-1))
00%
Ox
alou((fk)
Dlnn (8%-1)
0h-1 - (N
+mm)emme
cm
30%
aNm
00% k-1
Olou((%)
Hlin (0%-1)
( (Xp))emee eme
k-1
00%
2Xm
axm
00 k-1
(luu((f))
Hein (0k-1)
0(2
k- Cm
00%
00k_
aXh
Olin (0)
(taking
as constant w.r.t Nin to get the first-order MAMT.
00
approximation as in
30 16)
(A.14)
(loue(1%)
o
em
00%
ax
002-1
0(xm)+ is equal to identity using approximations from trraight-througe ON
estimation as in (5) 45
Olou((2))
(0/-1)
(0)
e
00%
00 k-1
aloue(P))
Oein (0
0g
e
30%
302
===0
(repcatedly expanding and differentiating the update function U())
Imeta (Min) = alou((f)
k-1
(ll.
Cm
00%
a0k
===
(assuming initial parameters, 0g time j is constant w.r.t X
This meta-gradient is used in Equation g (main paper) for scale updates.
Next to obtain the meta-gradients of (weight) parameters, of lets recall Equation A.11
① (0). A 0lnn(0)) 00,
Imeta (0)
II ( 9K ===
II k==
(A.15)
002
gk
Setting all the first-order terms constant in the right-hand side of this equation (to ignore the second-order derivatives), we get the first-order approximation of the meta-gradient as:
F (0)) = Ok = (o)))
(A.16)
which is used model updates in Equation 4 (main paper). This approximation drastically reduces memory consumption while preserving the performance. This allows scaling of Amphibian the larger networks with complex datasets (Table 2 (main paper)).
3

---

Published in Transactions on Machine Learning Research (04/2025)
3 Amphibian Algorithm
The pseudocode of the Amphibian algorithm is provided in Algorithm [ in the main paper. For the given online batch of data, B,, Amphibian performs inner-loop updates, where in each update uses a sample (or a subset of samples) from B, (Line 8). On the final parameters obtained after inner-updates, 0% evaluates meta-loss on the entire batch, B; to obtain meta-gradients for scales, V four (fu; BB)) and weights, og lout (6%; B.). With this meta-gradicnts, Ampibian first update the scales (Linc 10) and then model weights (Line 11). Tn inner-updates (Tine 8) and outer-update (Line 11), to avoid gradient ascent, only positive parts of the scales are used using: (Am)= 20mm function, where 1.20 IR + {0, 1}. For bias parameters in each layer, use À vectors instead of diagonal A matrices in inner-updates (Line 8) and outer-update (line 11). Each element of X, learns the learning rate of the corresponding bias parameter which is updated using the similar update step in Line 10.
4 Experimental Details
4.1 Dataset Splits and Statistics
Split CIFAR-100 has 20 tasks each having distinct classes from CIFAR-100 211. Split TinyImagenet has 40 lasks where each task has 5 distinct classes from T'nyImagenet-222 5-Datasets including CIFAR-10, MNIST, SVHN, Fashion MNIST, and notMNIST where each dataset is split into five tasks (cach having a 2 classes) to obtain a total of 25 tasks in the split 5-Datasets sequence. Dataset statisties used in these experiments are given in Table A.1 and A.2 Split minilmageNel (40) consists of splitting the minilmageNet dataset (44) into 10 disjoint tasks, where each task contains 10 classes. Here each image of size 3 x 84 x 84. ImageNNet-100 dataset. Split ImageNet-
(22). Finally, use a sequence of
built by selecting 100 classes from the ImageNet-Ik (I2) which used in our experiment, consists of splitting the ImageNet- into 20 disjoint tasks, where each task contains 5 classes. Here each image is of size 3 x 224 x 224. For both split minilmageNet and ImageNet-100, 2% training data from each task is kept aside as validation sets.
Table A.1: Dataset Statistics. 10% training data from each task is kept aside as validation sets.
Split CIFAR-100
Split Tinyimagenct
Split 5-Datasets
mum. of tasks
20
40
25
input size
32 x 32
64 x 64
x 32
32
Classes/task
5
5
# Training samples/tasse tasks
2,250
2,250
See Table A.2
# Validation Samples comples/taslee taases
250
250
See Table A.2
Test samples ttakks /tasks
500
250
Table A.2
Table A.2: 5-Datasets statistics B6 For the datasets with monochromatic images, we replicate the image across all RGB channels so that size of each image becomes 32 32. split 5-Datasets, each dataset (in this table) is split into tasks, each with 2 disjoint classes. 10% training data from each task is kept aside validation sets.
CIFAR-10
MNIST
SVHN
Fashion MNIST
notMNIST
Classes
10
10
10
# Training samples
45,000
54,000
65,931
54,000
15,167
Validation Samples
5,000
6,000
7,325
6,000
1,685
# Test samples
10,000
10,000
26,032
10,000
1,873
4.2 Network Architecture Details
For split CIFAR-100 experiments, similar La-MAML (16), we used a 5-layer neural network with 3 convolutional layers each having 160 filters with 3 kernels, followed by two fully connected layers having
4

---

Published in Transactions on Machine Learning Research (04/2025)
320 units each. For split 5-Datasets, used 5-layer neural network with 3 convolutional layers each having 200 filters with 3 x 3 kernels, followed by two fully connected layers having 400 units each. For split Tinylmagenet experiments, similar to La-MAML (G), we used a 6-layer neural network with 4 convolutional layers cach having 160 filters with x kernels, followed by two fully connected layers having 640 units each. For ImageNee-100 and minilmageNet experiments, we have used Res.eet-18 model. This network consists of a front convolutional layer followed by 4 residual blocks each having four convolutional layers followed by classifier layer. We used 40 filters in front convolutional layer and in the first residual block layers. For second, third and fourth residual blocks used 80, 120 and 160 filters respectively. For ImageNet-100, in the front convolution layer, we used convolution with 7 x 7 kernel with stride 5. For minilmageNet, in the front convolution layer, we used convolution with 3 x 3 kernel with stride 2. For both of these cases, we used 2 x 2 average-pooling with stride 1 before the classifier layer. All the networks use ReLU in the hidden units and softmax with cross-entropy loss in the final layer.
4.3 List of Hyperparameters
A list of hyperparameters our method and baseline approaches is given in Table A.3 As in (6). hyper- parameter for all the approaches are tuned by performing a grid-search using validation sots. For all the experiments, except. split ImageNet-I000 a batch size of 10 was used for training. In split. Tee experiments batch size of 25 samples was used.
4.4 Baseline Implementations
For rehearsal- based methods ER, GEM, A- A-GEM, MER and La-MAML, we used the implementation pro- vided in La-MAML (16). CLS-ER and DER++ are implemented by adapting the codes by (3). EWC (20).
SI (D)) and NCL (19 are implemented adapting the codes from GPM (66) and SGP 37 are implemented
using the respective official open-sourced code repositories.
4.5 Amphibian Implementation: Software, Hardware and Code
We implemented Amphibian in python (version 3.7.6) with pytorch (version 1.5.1) and torchvision (version 0.6.1) librarics. We the codes single NVIDIA TITAN Xp GPU (CUDA version 12.1) and reported the results in the paper.
httt:://sthhaaamon/aaaaaaaeeeaie (MIT License)
5

---

Published in Transactions on Machine Learning Research (04/2025)
Table A.3: Hyperparameters grid considered for the baselines and Amphibian. The best values are given in parenthescs. Here, 'Ir' represents the learning rate. All the methods use SGD optimizer unless otherwise stated. The mumber of epochs in the OCL setup for all methods is 1. To maximally utilize the current batch of data in OCT, each method has hyperparameter called glances (I6) 51) which indicates the number of gradient updates or meta-updates made on each of these batches. In the table we represent Split CIFAR-100 as 'cifar', Split Tinylmagenet as 'tinying' and Split 5-Datasots as '5data'.
Methods
Hyperparameters
ER
Ir 0.01 (5data), 0.03 (eifar), 0.1 (tinying): glances (5data), 10 (cifar, tinying) memory size (data samples) 100 (5data), 200 (eifar), 100 (tinying)
GEM
Ir 0.01 (5data), 0.03 (cifar, sinying): glances 1 (5data), (eifar, tinying) menorry size (dala samples) : 100 (5dalaa), 200 (cifar), 400 (tinying)
A-GEM
lr 0.01 (tinying, 5data], 0.03 (eifar); glances: 1 (5data), (elfar, tinying) memory size (dala samples) 100 (5data), 200 (cifar), 400 (tinying)
MER
Ir (ax) 0.05 (5dalu), D.1 (cifar, tiuying)
Ir (9) 0.1 (cifar, tinying, Sduta)
Ir (7)) 1.0 (eifar; sinying, 5data)
glances: (5data), (cifar, tinying) memoory size (dala samples)
(5duta), 200 (cifar), 400 (tinying)
DER +++
Ir 0.01 (5data), 0.03 (elfar), 0.1 (tinying) glances 1 (5data), 2 (tinying), 10 (clfaz)
o: 0.1 (tinying), 0.2 (cifar, 8data) ; 8: 0.5 (eifar, tinying). 1.0 (5data) memory size (data samples): : 100 (5data), 200 (cifar), 400 (tinying)
CLS-ER
Ir 0.01, 0.03, 0.05 [cifar, linying, 5data) glancss: 1 (Sdataa), 2 (tinying). 10 (cilar) r.: rgt 0.3 (Sdataa), 0.5 (cifar), 0.9 (tinying) ; rye 0.5 (cifarr, 0.8 (tinying), 1.0 (5data)
A: 0.1 (eifar, tlyying, 5data), 0.15
memory size (data samples) 100 (5data), 200 (eifar), 400 (tinying)
Le-MAML
Oo 0.1 (cifar, tinying, 5data)
(v)) 0.25 (5data), 0.3 (cifar, tinying)
glances: (5data),, 2 (tinying), (rifar)
memory size (data samples): 100 (Sdata), 200 (eifer), 400 (sinying)
EWC
Ir 0.1 (ctfar, tinying, 5data)
regularization coefficient, x 1e", le (elfar), Ie 2e (5dats), le2 (tinying) glances 1 (5data), 2 (thuying). 5 (cifax)
SI
optimizer: Adam (cifer, tinying, 5data)
Ir le 3 (cifar, linying, 5dataa) regularsation coefficient,
50 (sinyinng, 5data), 100 (cifar), 1000
glances: 1 (5dataa,, 2 (tinying), 5 (ciffr)
GPM
Ir: 0.05, 0.1 (ciforr, tinying, 5data)
fax : 0.96 (tinying), 0.975 (Edsta). 0.98 (eifar)
fex dincreeent/tssle : 0.001 (cifar, tinying, 5dats)
n,, 120 (cifar, tinying. 5data)
glances: (5data),, 2 (tinying), 5 (cifar)
SGP
lr 0.05, 0.1 (cifar, Linying, 5dala)
0.96 (tinying), 0.975 (5dataa), 0.98 (tifar) (Increment//tase scale coefflcient (ax) 1 (5data), (cifar), 10 (tayying) 120 (cilar, tinying. 5data)
0.001 (elfaz, thnynng, 5data))
giances: (5data), 2 (tinying), 5 (cifar)
NCL.
Ir 0.05, 0.1 (cifar, 5data), (sinying)
PC2 2250 (cifar, tinying), 9000 (5data)
glances: 1 (5data), 2 (Luyingg), 5 (cifax)
Amphibian
A² 0.1, 0.25 (tinying). 0.5 (cifar, 5data)
Ir (1)) 0.25, 0.5 (tinying). 1.0 (cifar, 5data)
glances: 1 (5data), 2 (tinying), 5 (cifax)
6

---

Published in Transactions on Machine Learning Research (04/2025)
5 Additional Results
5.1 Memory Overhead and Training Time
In Figure A.1 a) we show memory overhead comparisons during training for 5-Datasets tasks. In this case also, we observe that other baseline methods have orders of magnitude more memory overhead than Amphibian. Wall-clock training time comparisons among different methods for all three datasets are shown in Figure A.I Training times for all the tasks in the continual learning sequence for different experiments are measured on single NVIDIA TITAN Xp GPU. As Amphibian uses inner-and outer-loop meta-learning steps, it requires more wall clock time for each model update during training compared to the other rehearsal- free basclines. However, other rehearsal-based meta-learners such as La-MAML, and MER take up to ~2.77 and ~00 more training time than Amphibian.
(a)
(b)
Training Time (Normaiized)
Memory (Normalizedl
1
10a
10a
100
10
5-Datasets
CIFAR-100
TinylmageNet
5-Datasets
EWC
La-MAML
Amphiblan
MER
Figure A.1: (a) Comparison of memory overhead (normalized by the Amphibian memory overhead) during training for 5-Datasets experiments. (b) Wall-clock training time comparisons for sequential training of all the tasks in different datasets. Normalized with respect to the time taken by Amphibian.
(a) Task 2 test loss
(b) Task 7 test loss
(c) Task 11 test loss
>
2
>
2
>
2
0
2
2
0
1.0
1.0
1.0
1.4
1.4
1.5
0.5
0.5
1.2
0.5
1.2
0.0
0.0
0.0
0.7
4
11
9.0
-055
0.5
-0.5
0.5
0.5
0.5
0.5 0.0
0.5
1.0
0.5
0.0
0.5 1.0
0.5
0.0
0.5
1.0
Figure A.2: Dynamics of continual learning in Amphibian. Loss contours of (a) Task 2, (b) Task 7, and
(c) Task 11 from split TinyImagenet dataset are plotted on 2D planes defined by parameters (02, 020 840), (07, 0 020) and (0'1,00 025,000 respectively. Black lines indicate learning trajectories.
5.2
Continual Learning Dynamics in Amphibian: Loss Landscape View
Loss Contour Plots. We used visualization tools developed in 20 [3 to plot the loss contours (in Figure 0 (main paper) and A.2 on 2D planes defined by model parameters (0'). Each of these hyperplanes in the parameter space is defined by three points 01, 8 and Ø3 Orthogonalizing Ø 01 and Ø - gives a two dimensional coordinate system with base vectors u and v. The value at point (x, v) is then calculated as the loss of model with parameters 01 + I + Please see the code/appendix in (29) for more details.
Figure D (main paper) we showed that Amphibian updates incur minimum no increase in losses of the past tasks for split CIFAR-100 tasks. Here, Figure A.2 we show the loss contours for three split TinyTmagenet tasks. In Figure A.2 a) plot the loss
Continual Learning Dynamics in Amphibian.

---

Published in Transactions on Machine Learning Research (04/2025)
contour of task 2 from Tinylmagenet in 2D plane. This figure shows when we sequentially learn from task 2 to task 40, along the entire learning trajectory (02 + 0 + 020) loss of task 2 only increases minimally from the initial point (0²). A similar trend is also observed for other tasks. Here we show such trends for task 7 (Figure b)) and task 11 (Figure c,2
Table A.4: Number of meta-learnable gradient scales (in diagonal matrix, A) in cach layer in Amphibian.
Network
Layer
Size of Filters Weights
Number of Scales (in Scale Matrix, A)
(C, x C, x le x k) (m x n)
G-layer Network
Couvl
160 x 3 3
27
(CTFAR-100)
Conv2
160 160 3 3
1400
Couv3
160 160 3 x
1400
FCI
320 x 2560
2560
FC2
320 x 320
320
G-layer Network
Conv1
160 x 3 x 3 x 3
27
(TinyImagenet)
Conv2
160 160 3 x
1400
Couv3
160 160 x 3
1400
Couv4
160 160 3
1400
FCI
640 2560
2560
FC2
640 x 640
640
5-layer Network
Convl
200 3
(5-Datasets)
Conv2
200 x 200 3 x 3
1800
Couv3
200 200 x
1800
FCI
400 x 3200
3200
FC2
400 x 400
400
5.3 Number of Gradient Scales in Amphibian
As we adopt low-dimensional gradient space representation (as discussed in Section 3 (main paper)) for gradients of weights /fltter in the neural network, the number of learnable scales for gradient bases (in scale matrix, A) in Amphibian is very small compared to the size of the bhights/filters Table A.4 shows the number of meta-learnable scales for each layer. Such a small number of scales explains the extremely low memory overhead of Amphibian during training as shown in Figure 4(c) (main paper) and Figure A.1 (a.
5.4 Analysis of A During Training
To demonstrate the inner workings of the proposed algorithm, in Figure A.3 we plot the layer-wise evolution of the A during training of 20 CIFAR-100 tasks. For each layer, we show a box plot of À values contained the diagonal matrix A. In Figure A.3 (e)) the horizontal red dotted line indicates 0 value, whereas Task Id 0 indicates model initialization point where all the À have positive value of 0.5. In Figure A.3 ()) this corresponds to 100% active gradient directions, meaning all the gradient directions are available for gradient descent. As we learn more and more tasks, across the layers, more X becomes negative. In our algorithm, to ensure the stability of the old knowledge (hence less forgetting), we avoid taking gradient steps along the directions with negative X. This trend is captured in the Figure A.3 f)) where, as we learn more tasks, the total number of (active) gradient directions along which optimization steps can be taken is gradually reduced. This finding is in agreement with the previous findings in the EWC 200 (where higher mumber of weight changes are heavily penalized as more tasks learned continually) and GPM (36) (where the number of active gradient directions is reduced as more tasks are Icarned). The main difference is in those methods an explicit constraint or regularization is used to control stability--lasttiity of the model for continual learning, whereas in Amphibian that constraint is learned implicitly from the data. Figure A.3 ()-(e) also reveals that at different layers different level of constraints are needed for online continual learning and it would be hard to hand design them explicitly. This is where learning from data offers the most benefits compared to the other methods in achieving optimal performance for fast online continual learning.
8

---

Published in Transactions on Machine Learning Research (04/2025)
(a) Layer
(b) Layer 2
200
A
600
-1000
Task ld
Task
(c) Layer
(d) Layer 4
A
A
-10
Task ld
Task ld
(e) Layer 5
Active gradient directionn (96)
1.00
-50
A
-20
40
39 20
Task ld
Task ld
Figure A.3: (a)-(e) Layer-wise evolution the during training of 20 CIFAR-100 tasks. For each layer, we show box plot of À values contained in the diagonal matrix A. The horizontal red dotted line indicates 0 value and Task Id 0 indicates the model initialization point where all the À are 0.5. As we learn more tasks more À in each layer become negative. (f) Percentage (%) of active gradient directions after learning each CL. task.
6 Representation of the Gradient Space in Neural Network
Here we elaborate on the point that SGD (gradient) updates subspace spanned by the input data points. Following (36), let's consider a single layer linear neural network supervised learning setup where æ e R is the input vector, y E Rm is the label vector in the dataset and W e mmxx are the parameters (weights) of the network. The network is trained by minimizing the following mean-squared error loss function
L -=
W - y
(A.17)
We can express gradient of this loss with respect to weights as
V L. (Wa) x y) 8x
(A.18)
9

---

Published in Transactions on Machine Learning Research (04/2025)
where ò E R is the error vector. Thus, the gradient update will lie in the span of input (æ)), where elements in 8 scale the magnitude of æ by different factors. Here, we have considered per-example loss (batch size of 1) for simplicity. In the batch/ming m/mmmminibbaat baini-batch setting, the batch loss is the summation of the losses due to individual examples, the total batch loss for samples can be expressed as
k
k
Lhatch
-44=
L
Wa: x; - y: y:
(A.19)
The gradient of this loss with respect to weights can be expressed
V WLatch = 8 o, + 5222 + + 54.2%
(A.20)
Therefore, the gradient update will remain in the subspace spanned by the input examples. deep neural networks, the number of imput examples is very large compared to the dimension of the inputs, k >> n. Assuming this, the largest subspace these k input examples can span will be n dimensional.
In this example, since weight W is m n dimensional, naturally the corresponding gradients are m n dimensional. However, according to Equation A.20 gradients will lie in the n dimensional subspace. So, in the paper, when we refer lo: "the gradients lie in a low dimensional subspace", we imply gradients will lie in the dimensional subspace within the original m X dimensional space. With the one hot vector e, we defined the canonical basis of that subspacc. Following the definition in (36), we denoted that space as gradient space. We can draw the same conclusions by analyzing the convolutional layers the neural netwrrks, for that we refer to Section (36).
10


Reviews:
[{"review_id": "wkYppdk65Z", "reviewer": "Reviewer_oruh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. The problem of replay-free continual learning is important\n2. Empirical results are comprehensive\n3. Most of the paper is well-written and easy to follow\n\n## Weaknesses/Questions\n1. Some parts of the paragraph **Representation of the Gradient Space in Neural Network** are not explained adequately.\n- The line: \"Since SGD updates lie in the span of input data points (48), gradients (or gradient space) in each layer of the ANN can be represented by low- dimensional basis vectors (34).\" could use further explanation. Concretely, I am not sure how the first part motivates the second\n- How is the dimension of the gradient space chosen? For a Conv layer, why can't we describe the gradients in $(C_o \\times k \\times k)$ space? How is this decided? \n\n2. Is it necessary to learn $\\Lambda$? What happens if we instead use optimizers that have built in ways of scaling the gradient updates i.e RMSProp or Adam?\n\n3. In Table 1, it seems very counterintuitive that the methods that use replay underperform the methods that do not use replay? How were replay examples chosen, and what is the buffer size?\n\n4. How would the authors modify Amphibian to be able to use samples in a replay buffer in settings where this is available?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Amphibian, a gradient-based meta-learning framework for replay-free continual learning. Amphibian adapts quickly to new data while avoiding forgetting by learning to scale gradient directions through a meta-objective that aligns gradients.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weaknesses/Questions", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tcrAHO78ek", "reviewer": "Reviewer_DSRc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n* The paper is well written and clearly defines what it aims to do and how\n* The method works well and is tested in a variety of different settings, as well as compared to relevant baselines. \n\n### Weaknesses\n* The main concern I have with this work is that the inner-outer loop updates take multiple forward passes to perform one single update. This requires a lot more compute than a simple forward pass, and by performing forward passes at multiple different weights, it can extract more information from the current batch than the the other tested baselines can. It think it would be fair to at least have an experiment where some baseline can perform multiple updates on the same batch as well, so that they use the same amount of computational resource as the proposed method. Especially in an online setting as in this paper, having a few more update steps may be a crucial difference, as the networks are likely not converged to a minima after a single pass through the data. \n\n* The idea of the method is to encourage gradients to allow with the directions in which past gradients pointed. I wonder whether this isn't too restrictive. The loss landscape of a neural network is highly non-linear, which means that if a gradient pointed in one direction at one set of parameters, it may point in a completely different direction for another set of parameters. With this in mind, it is unclear why one should try to encourage gradients to keep pointing into similar directions during all of training. An analysis of the final $\\Lambda$ and its different versions throughout training would be a nice additional experiment to better understand what the method is doing. \n\n* I found the mathematical derivation of the method quite hard to follow. Especially how the gradient base vectors are made from the matrix $\\Lambda$ is not clear to me. They are called \"selective gradient direction\", but I could not find how these directions are defined. I think the derivation in section 4 can be made a bit clearer, although I understand that meta learning update rules are not easily described in a concise way.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a meta learning objective for rehearsal free online continual learning. It uses subsets of the current mini batch to perform inner-loop updates, and the full mini batch is used for the outer loop update. The outer loop update updates both the model parameters and a scaling matrix for the gradient. The scaling matrix is a diagonal matrix which learns the learning rate of different layers. Instead of using a full diagonal matrix, it is noted that the gradients of a single layer can only span a subspace as large as the input dimension, which is used to reduce memory footprint of the method. Experiments on various task-incremental benchmarks are performed, comparing to both rehearsal and rehearsal free methods. Both continual learning and meta learning methods are compared using their final accuracy, average forgetting (BWT) and learning efficiency. Finally, an experiment is performed that controls the strength of the update, which shows that other baselines may be too static and not learn anything at all.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* (For acceptance): at least an analysis and comparison of the computational complexity of this method, preferably a comparison to another benchmark that uses the same amount of compute. \n* (For acceptance): a clearer explanation of how the base vectors $e_m$ are defined. \n* (To strengthen): An analysis of $\\Lambda$ throughout training.\n* (To strengthen): Making the derivation a bit easier to follow.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "RCM6PFM56d", "reviewer": "Reviewer_D2Ti", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The paper is well presented with clear writing and good use of easy-to-understand figures.\n- The proposed method achieves strong performance in continual learning. It beats many methods using rehearsal while being rehearsal free itself.\n\n## Weaknesses\n- The empirical evaluation compares to baselines that are few years old. The paper does not discuss whether there have been any advancements in online continual learning since those methods were published. Since the paper suggests that the proposed method achieves state of the art, it is important to accurately describe the state of the field in a self-contained way.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. **Introduction of Amphibian**: A meta-learning model for continual learning without rehearsals. The proposed method is a variant of MAML with a learned pre-conditioning matrix.\n2. **Novel Meta-Objective and Gradient Scaling**: The paper proposes a meta-objective that aligns gradient directions, aiming to optimize both speed and stability in learning.\n3. **High Performance on Benchmarks**: The model is tested on various continual learning tasks, demonstrating good accuracy in continual learning while efficiently learning with less data. It also shows strong few-shot learning abilities, allowing it to transfer knowledge effectively to new tasks.\n4. **Empirical analysis of the proposed framework**: Investigates the relationship between fast learning an forgetting via a regularized variant of the proposed method. Illustrates the behavior of the algorithm by visualizing loss landscapes.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- \"6.4 Understating Continual Learning Dynamics in Amphibian\" -> \"6.4 Understanding Continual Learning Dynamics in Amphibian\"\n- Include an explanation of why it is ok to mainly compare to older baselines.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is well written and is easy to read.
Initially reviews were split. Upon request for revision, the authors made efforts to revise the paper well and to eventually resolve all of their critical concerns.

Here are some minor things that should be considered for the final version.
- An earlier idea for learning a scaling matrix in the gradient-based meta-learning framework can be found in T-nets (Lee and Choi, 2018).
Yoonho Lee and Seungjin Choi (2018), "Gradient-based meta-learning with adaptive layerwise metric and subspace," ICML.
The  network architecture and its purpose might be different, but it would be helpful to briefly mention what was there and how it is different from each other.
- Meta-learning a diagonal scale matrix seems to reflect the alignment history. This is the key contribution in this paper. It would be nice to allocate more space for explaining it, since there is no any theoretical analysis to support it. For instance, please elaborate Fig. 2 as well as the last paragraph in page 5.
Decision: Accept (poster)