Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning
Authors: Max Vladymyrov
Andrey Zhmoginov
Mark Sandler
Venue: Accepted by TMLR ()

Abstract: We focus on the problem of learning without forgetting from multiple tasks arriving sequentially, where each task is defined using a few-shot episode of novel or already seen classes. We approach this problem using the recently published HyperTransformer (HT), a Transformer-based hypernetwork that generates specialized task-specific CNN weights directly from the support set. In order to learn from a continual sequence of tasks, we propose to recursively re-use the generated weights as input to the HT for the next task. This way, the generated CNN weights themselves act as a representation of previously learned tasks, and the HT is trained to update these weights so that the new task can be learned without forgetting past tasks. This approach is different from most continual learning algorithms that typically rely on using replay buffers, weight regularization or task-dependent architectural changes. We demonstrate that our proposed Continual HyperTransformer method equipped with a prototypical loss is capable of learning and retaining knowledge about past tasks for a variety of scenarios, including learning from mini-batches, and task-incremental and class-incremental learning scenarios.

Published in Transactions on Machine Learning Research (05/2024)
Continual Tranffrraso::mr:: A Meta-Learner for Continual Few-Shot Learning
Max Vladymyrov Google Research
mav@ggoogle.co
Andrey Zhmoginov
dchmogiin com
Google Research
Mark Sandler Google Research
sandler@google.com
Reviewed on OpenReview: https: // openreview. net/ forum? id= adtSq2nkm.
Abstract
We focus on the problem of learning without forgetting from multiple tasks arriving sequen- lially, where each lask defined using a few-shot episode of novel or already seen classes. We approach this problem using the recently published HYPERTRANSFORMER (HT), Transeormer-based hypernetwork that generates specialized task-specifie CNN weights di- rectly from the support set. In order to learn from continual sequence of tasks, propose to recursively re-use the generated weights as input to the HT for the next task. This way, the generated CNN weights themselves act as a representation of previously learned tasks, and the HT is trained to update these weights so that the new task can be learned without forgetting past tasks. This approach different from most continual learning algorithms that typically rely on using replay buffers, weight regularization or task-dependent architectural method
changes. We demonstrate that our proposed CONTINUAL equipped with prototypical loss is capable of learning and retaining knowledge about past tasks for variety of scenarios, including learning from mini-batches, and task-incremental and dlass-incremental learning scenarios.
1 Introduction
Continual few-shot learning involves learning from continuous stream of tasks described by small number of examples without forgetting previously learned information. This type of learning closely resembles how humans and other biological systems acquire new information, as we can continually learn novel concepts with a small amount of information and retain that knowledge for an extended period of time. Algorithms for continual few-shot learning can be useful many real-world applications where there is a need to classify a large number of classes in a dynamic environment with limited observations. Some practical applications can include enabling robots continually adapt to changing environments based on an incoming stream of sparse learning, where the model can be trained sequentially on
demonstrations or allowing for private data sharing only the weights without ever exposing the data.
To tackle this problem, we propose using EYPERTRANNSORREES (HT; Zhmoginov et al. 2022), recently published few-shot learning method that utilizes a large hypernet (Ha et. al. 2016) to meta-learn from episodes sampled from a large set of few-shot learning tasks. The HT is trained to directly generate weights of much smaller specialized Convolutional Neural Network (CNN) model using only few labelod examples. This works by decoupling the domain knowledge model (represented by Transformer; Vaswani 2017) from the learner itself (a. CNN), generated to solve only given specific few-shot learning problem.

---

Published in Transactions on Machine Learning Research (05/2024)
We present modification to HT method, called CONTINUAL HYPERTRANSFORMER (CHT), that is aimed at exploring the capability of the HT to sequentially update the CNN weights with the information from a new task, while retaining the knowledge about the tasks that were already learned. In other words, given the CNN weights ft-1 generated after socing some previous tasks O,...,t- and a description of the new task i, the CHT generates the weights 0g that are suited for all the tasks O,...,I.
In order for the CHT to be able to absorb continual stream of tasks, we modified the loss function from cross-entropy that was used in the HT to a more flexible prototypical loss (Snell et al.) 2017), that uses prototypes as a learned representation of every class from all the tasks. As the tasks come along, we maintain and update a set of prototypes in the embedding space. The prototypes are then used to predict the class and task attributes for a given input sample.
We evaluate CHT in three realistic scenarios where a continual few-shot learning model like ours might be usod: the mini-batch version, where every task consists of the same classes; the lifelong learning version, where classes for all the tasks are drawn from the same overall distribution; and the heterogeneous task semantic version, where every task has its own unique distribution of classes.
We also test CHT in two different continual learning scenarios: task-incramental learning (prodicting class attributes using the task information) and class incremeett learning (predicting class attributes without access to task information; also known as lifelong learning). Moreover, we show empirically that a model trained for itassincremmntal learning can also perform well in task-incremental learning, similar to a model specifically trained for tast-inerrmental learning.
Our approach has several advantages. First, as hypernetwork, the CHT is able to generate and update the weights of the CNN on the fly with no training required. This is especially useful for applications That require generating a lot of custom CNN models (such as user-specific models based on their private data). A trained Transformer holds the domain world-knowlede and can generalize from limited few-shot obscrvations.
Second, we did not observe catastrophic forgetting in CHT models when evaluating them on up to 5 tasks using OMNIGLOT and TEREDIMAGENET datasets. We even see cases of the positive backward transfer for smaller generated CNN model, where the performance on given task actually improves for subsequently generated weights.
Third, while the CHT is trained to optimize for T' tasks, the model can be stopped at any point t S T during the inference with weights 6g that are suited for all the tasks 0 < r I t. Morcover, the performance of a given weight O, improves when the CHT is trained on more tasks T.
Finally, we designed the CHT model to be independent from specific step and operate as a recurrent system. It can be used to learn larger number of tasks it was originally trained for.
2 Related work
Few-shot learning Many few-shot learning methods can be divided into two categories: metric-based learning and optinization-based learning. First, metric-based methods {Vinyals et al. 2016 Snell et al., 2017 Sung et al.] 2018 Oreshkin et al. 2018)) train a fixed embedding network that works universally for any task. The prediction based on the distances between the known embeddings of the support set and the embeddings of the query samples. These mothods are not specifically tailored for the continual lcarning problem, since they treat every task independently and have no memory of the past tasks.
Second, iitimmizaaiion-bssed methods (Finn et al. 2017 Nichol & Schulman 2018 Antoniou et al. 2019 Rusu ct al. 2019) propose to learn an initial fixed embedding, which is later adapted to a specific task using few gradient-based steps. They are not able to learn continually, as simply adapting the embedding for a new task will result in the catastrophic forgetting of previously learned information. In addition, in contrast to these methods, our proposed CHT generates the CNN weights directly on the fly, with no additional gradient adaptations required. This means that for a given task we do not have to use second-order derivatives, which improves stability of our method and reduces the size of the computational graph.
2

---

Published in Transactions on Machine Learning Research (05/2024)
arg min Et= (1(b
Task 0)
Task T
Jo(v) Lo (Ja_ce ). 07
Jr((b = Et- Lu (fo ),
Hyper Transformer
CNN Sa, (a(10)
Hyper Transformer
CNN (Joz (2())))) )
80 ag (S 0)
80
01-1
0r ag (5 Ar-1) Or
5th =
(Arg <Mt 1am0
(00) NK
Q
(0)
8.
(00
00
Support Set, task (
Query Set, task
Support Set, task T
Query Sel, tasks O...
Figure 1: In continual few-shot learning, the model learns from tasks sequentially. For the first. task (lask 0) the CNN weights Ou are generated using only the support set 5(() For cach subsequent task t, the CONTINUAL HYPORTRANSPORMER (CHT) uses the support set S(1) and the previously generated weights 0--1 to generate the weights 9.. To update the weights v of the CHT, the loss is calculated by summing the individual losses computed for each generated weight 6, when evaluated on the query set of all the prior tasks (Q(+))==--
Continual learning Most continual learning methods can be grouped into three categories based on their approach preventing catastrophic forgetting when learning a new task: rehearsal, regularization and architectural (see Pasunuru al. 2021; Biesialska al. 2020 for an overview). Rehearsal methods work by injecting some amount of replay data from past tasks while learning the new task (Lopez- Paz & Ranzato 2017: Riemer et al. 2018 Rolnick et al. 2019 Gupta et al.) 2020: Wang et al.] 2021a) or distilling a part of a network using task-conditiones embeddings Mandivarapu et al. 2020 Von Oswald et al.) 2019
Regularization methods introduce an explicit regularization function when learning new tasks to ensure that old tasks are not forgotten (Kirkpatrick et al.) 2017 Zenke et al.] 2017). Architectural methods modify the network architecture with additional task-specific modules (Rusu al. 2016), ensembles (Wen et al. 2020)
or adapters (Pfeiffer al. 2020) that allow for separate routing of different lasks.
We believe that our approach requires the least conceptual overhead compared to the techniques above, since it does not impose any additional explicit constraints to prevent forgetting. Instead, we reuse the same principle that made HT work in the first place: decoupling the specialized representation model (a CNN) from the domain-aware Transformer model. The Transformer learns how to best adapt the incoming CNN weights in a way that the new task is learned and the old tasks are not forgotten. In this sense, the closest analogy to our approach would be slow and fast weights (Munkhdalai & Yu 2017). with the Transformer weights being analogous to the slow weights that accumulate the knowledge and generate CNN weights fast weights.
Incremental few-shot learning A related, but distinct area of research is incremental few-shot learning
GGidaris & Komodakis 2018 Ren et al. 2019 Perez-Rua et al. 2020 Chen & Lee 2020 Tao et al.) 2020
Wang et al. 2021b Shi et al. 2021 Zhang et al.] 2021 Mazumder et al.) 2021 Lee et al. 2021 Yin et al, 2022). There, the goal is to adapt few-shot task to an existing base classifier trained on a large dataset,
without forgetting the original data. The typical use case for this would be a single large classification model that needs to be updated once in while with novel classes. These methods do not work when the existing base classifier is not available or hard to obtain. Another issue is applicability. These methods have to be retrained for each new sequence of tasks, which severely limits their practical application. In contrast, CHT is meta-trained on a large public set of many few-shot classes. After training, it can generate task-dependent 0 in a matter of seconds for many novel sequences.
Consider the following use-case scenario. There a large public database of many few-shot classes available on the server, and a large number of users with private sequential few-shot data. There is a whole range of applications one can think of in this space, for example smartphone hotword detection or robot calibration
3

---

Published in Transactions on Machine Learning Research (05/2024)
of unique environments in users' homes. Incremental few-shot learning would not be able to either train reliable base classifier due to the large number of classes on the server, nor effectively deal with user's private data, as it needs to be re-trained each time on the users' devices, which is a lengthy process.
On the other hand, our CONTINUAL server (with as many classes as needed, as trains episodically and does not require a single monolithic classifier) and then shipped to devices. Each user can then generate or update their model on the fly without any training required, in matter of seconds. In the paper we have demonstrated the ability of the CHT to update the user's private classifiers with novel instances of already existing classes (Section 6.1 as well as new classes (Section 6.2 and 6.3) on the fly without retraining the system. This would be impossible or require lengthy retraining with any existing approach.
can be trained on a large public dataset on the
Perhaps the closest to our setting is the paper by Antoniou et al. 2020 which focuses on the general problem definition of the continual fow-shot learning, but falls short of providing a novel method to solve Another related method is Waann et al.) (2023) which proposes a heterogeneous framework that uses visual as well additional semantic textual concepts. In our paper only focus on visual input for model training and predictions.
3 Continual few-shot learning
We consider the problem of continual few-shot learning, where we are given a series of T lasks, where each task := {5(0, ) is specified via a K-way N-shot support set st() == (a y; N and a query set Q(+) == (a(", Di (1) NK where K is the number of classes in each task, N is the number of labeled examples J=
>
for each class, and N (typically N > N) the number of query examples to be classified.
We assume that the classes composing each individual task are drawn from the same distribution uniformly at random without replacement. However, we consider different ways in which classes for different tasks are chosen. First, cach task may include exactly the same set of classes. This is similar to mini-batch learning with T iterations, where each batch contains exactly N examples of each of K classes Second, each task might include a different set of classes, but drawn from the same overall distribution of classes. This corresponds to a lifelong learning scenario, where tasks can be thought as observations that allow us to learn more about the world as we encounter new classes during the inference. Finally, each task might have its own unique semantic meaning and the classes for different tasks are drawn from different distributions. We will evaluate all of these scenarios in our experiments.
Figure iilustrates the process of learning of a continual few-shot problem. For each of the tasks E O.....T. a learner 4p (parameterized by v)) needs to produce CNN weights O, based on the support set 5(4) of task and previously generated weights 0z-1 (except for the first task, where O, is generated only using 5(0));
0: := av
(1)
such that Og can predict the classes from all the tasks 0,...,t. f. Notice that when learning from task t, the learner does not have access to the support set of past tasks and must rely solely on the input weights f--1 as source of information from previous tasks.
After the weights 0₂ are generated, we can use the query set Q(7) of all tasks t E o,...,, t to evaluate the prediction quality of the 0g and calculate the loss Es with respect to the learner parameters vb In this work, we consider two lypes of predictions given the weights 0::
incremencrl learning, in which the goal is to identify the class attribute given the sample and its task attribute: p(ğ = k))
dlass-incrementl learning, in which the goal is to identify both class and task attributes of the samples: p(ū = k, 7())
This scenario does not require continual learning per se, as the classes do change between the tasks.
4

---

Published in Transactions on Machine Learning Research (05/2024)
HT
(Suited for task
CNN weights
CHT
CNN weights A (Suited for tasks 0...
Next task (t 1)
Emboddings
Weight slices
Weight slices
Sample Emheddings
Shared Weight Embeediings
Transformer/
Weight Generator
Transformer Weight Generator
Incoming Weight Slices
Discarded
Samples from task
Empty placcholders
Samples from task
Weight slices i
Generated Weight Slices
Previous task (t-1)
The information flow the HYPERTRANSEE TRANSFORMER (HT) model (left) compared to the proposed CONTINUAL TRANSFFRRRRR (CHT) model (right). In the original HT, the input weight embeddings are initialized with empty placeholders. In contrast, the proposed CHT model incorporates information from past tasks when generating weights for the current task. The weight slice information from previously learned tasks is passed as imput to the new iteration of the CHT. The CHT uses the support set for the current task and the input weight information to generate the weights. This allows the CHT to retain knowledge about past tasks and avoid forgetting when learning new tasks.
Figure 2:
Finally, we can test the performance of the trained model ag on episodes sampled from a holdout set of classes Crest- Notice that, in general, the total number of tasks for the test Teest might be different from T.
4 Continual TrapefTransormerr
Notice that for T' 1, the continual learning problem above reduces a standard few-shot learning problem defined by single few-shot learning task tu (S(0) QO)).. One method that has been effective in solving this type of problem is HYPER TRANSFORMER (HT, Zhmoginov et al. 2022 that uses self-attention mechanism to generate CNN weights 0 directly from the support set of the few-shot learning problem (see Figure 2
left).
We first describe the HT architecture. CNN weights are constructed layer by layer using the embeddings of the support set and the activations of the previous layer. After the weights have been generated, the cross-entropy loss Lu (Jo(E), f)) is calculated by running the query set (f, through the generated CNN.
To encode the knowledge of the training task distribution HT is using Transformer model that generates the resulting CNN layer-by-layer. The input of the Transformer consists of concatenation of image embeddings, activation embeddings and support sample labels. The image embedding is produced by a cnnvolutional feature extractor, this time shared among all the layers. The activation embedding is given by another feature extractor that is applied to the activation of the previous label (or the input, in case of the first layer). Together, the parameters of Transformer, image embedding and activation embedding constitute the HT parameters vb.
Our proposed CONTINUAL of tasks by using the generated weights from already learned tasks as input weight embeddings into the weight generator for a new task (see Figure 2 right). In this way, the learned weights themselves act as both the input and the output of the CHT, performing a dual function: storing information about the previous tasks as well as serving as the weights for the CNN when evaluating on tasks that have already been seen.
(CHT) naturally extends HT to handle continual stream
For each task t, the CHT takes as input the support set of that task ((t) well the weights from the previous tasks 0.-1, and generates the weights using the equation (1) that suited for all the tasks E O.....l. l. Therefore, for each slep I we want to minimize the loss on the query sets of every task up to t:
J((z) = Σ Lv (((
(2)
7=0
5

---

Published in Transactions on Machine Learning Research (05/2024)
The overall loss function is simply the sum of the losses for all tasks:
arg min J((e).
(3)
The CHT generates sequence of weights {0, [0-)+=0- such that each weight is suited for all tasks up to the current task: 8g performs well only task 0, 81 performs well tasks 0 and and so on. This allows the model to effectively learn and adapt to stream of tasks, while also maintaining good performance on previously seen tasks.
This design allows for a "preemptive" approach to continual learning, where the CHT model can be trained on tasks, and run for any number tasks т < T, producing well-performing weights 0, for all the tasks seen up that point. An alternative approach would be specify the exact number of tasks in the sequence in advance, and only consider the performance after the final lask T. This would correspond to minimizing only the last term (+) in the equation (3). However, in our experiments, we did not observe any significant improvement using this approach compared to the one we have described above.
Another desirable property of the proposed CHT architecture is its ability to be recurrent. The parameters of the HT do not depend on task information, and only take the weights & and the support set as input. This means that is not only possible to preempt CHT some carlier task, but also extend the trained model to generate weights for additional tasks beyond the ones it was trained. We will demonstrate this ability in the experimental section.
Algorithm Class-incremental learning using HYPER TRANSFFRRERR with Prototypical Loss.
Input: T randomly sampled K-way N-shot episodes: (8(1) Output: The loss value J for the generaled set of tasks.
1: J 1+0
> Initialize the loss. b Initialize the weights.
2: 8-
for 6 T do
O: x a (S(t), 0)-1)
Generate weight for current task. > Compute prototypes for every class of the current task.
for x 0 to K do
Cek N E (aa)estt fe, (x) 7: end for
8:
for T + 0 to t do
b Update the loss with every seen query set using the equation 6
9: for + 0 to K do
10:
J J+ J - ((.a)eo logp(0 = k.
11: end for
12: end for
13:
end for
4.1
Prototypical loss
The last element of the algorithm that we have left to discuss is the exact form of loss function (.) in the equation (2). The original HT used the cross-entropy loss, which is not well suited for contimual learning because the number of classes that predicts is tied to the number of parameters in the head layer of the weights 0. This means that the number of tasks increases, the architecture of CNN needs be adjusted, which goes against our design principle of using recurrent CHT architecture. Another option would be to fix the head layer to the K-way classification problem across all the tasks and only predict the class information within tasks (a problem known as domain-incremente this would cause classes with the same label but different tasks to be minimized to the same location in the embedding space, leading to collisions. Additionally, since class labels are assigned at random for each training episode, the collisions would occur randomly, making it impossible for CHT learn the correct class assignment. In the Appendix A..I we show that the accuracy of this approach decreases dranatically as the number of tasks increases and becomes impractical even for just two tasks.
lcarning; Hsu et al. 2018). However,

---

Published in Transactions on Machine Learning Research (05/2024)
To make the method usable, we need to decouple the class predictions of every task while keeping the overall dimensionality of the embedding space fixed. One solution is to come up with fixed arrangement of TK points, but any kind of such arrangement is suboptimal because it is not possible to place TK points equidistant from cach other in a fixed-dimensional space for large T. much more clegant solution is to learn the location of these class protolypes from the support set itself, e.g. with a protolypical loss (Snell ct al. 2017). The prototypes are computed by averaging the embeddings of support samples from a given class and task 7:
Crk = N
Σ fe,(r)1y=k-
(4)
(2.N)
We can use the prototypes in two different continual learning scenarios. First, for the lask-incremental learning, we are assumed to have access to the task we are solving and need to predict only the class information. The probability of the sample belonging to a class & given the task T is then equal to the softmax of the & distance between the sample and the prototype normalized over the distances the prototypes from all the classes from
exp(-I1fo, c=
p(0 x) = ΣN exp(-Il.o.. C+ 112)
(5)
Second, for more general dlas--nncrementll learning, we need to predict class attributes across all seen tasks. The probability of a sample belonging class & of task equal the softmax of the lz distance between the sample and the prototype, normalized over the distances to the prototypes from all classes for all tasks:
exp(- plū = k()) Σn' exp(-I15o, (+) Cr 122
c= 313)
(6)
The final loss function is given by minimizing the negative log probability of the chosen softmax over the query set. The pseudo-code for the entire CHT model is described in Algorithm [
Empirically, we noticed that the CHT models trained with the class-incremental learning objective (6) perform equally well in both class-ineremental and
settings, while models trained with the task-ineremennal objective 5 perform well only in the inseemanrettt setting and rarely outperform models trained with the equation (6. Therefore, we will focus on CHT models trained with the equation G and evaluate them for both lask- and class-incremeentl learning scenarios.
Notice that the prototypes are computed using the current weights 0, in the equation (4) for task T, but they are used later to compare the embeddings produced by subsequent weights 0e in equation (6). Ideally, once the new weights 0, are generated, the prototypes should be recomputed as well. However, in true continual learning, we are not supposed to reuse the support samples after the task has been processed. We have found that freezing the prototypes after they are computed provides a viable solution to this problem, and the difference in performance compared to recomputing the prototypes every step is marginal.
Finally, we want to highlight an important use-case where recomputing the prototypes might still be possible or even desirable. The weights O, are not affected by this issue and are computed in continual learning manner from the equation ( without using information from the previous task. The support set is only needed to update the prototypes through generated weights, which is relatively cheap operation. This means that it is possible to envision a srivacy-preservin scenario in which the weights are updated and passed from client to client in a continual learning manner, and the prototypes needed to "unlock" those weights belong to the clients that hold the actual data.
5 Connection Between Prototypical Loss and MAML
While the core idea behind the prototypical loss is very natural, this approach can also be viewed as a special case of simple 1-step MAML-like learning algorithm. This can be demonstrated by considering a simple classification model ( Φ) s(Wr)( b) with (W, 0), where fo(x:) is the embedding and s(-)

---

Published in Transactions on Machine Learning Research (05/2024)
is a softmax function. MAML algorithm identifies such initial weights 6 that any task with just few gradient descent steps initialized at Φ brings the model towards task-specific local optimum of Lr.
Notice that if any label assigmment in the training tasks is equally likely, it is natural for q(x; のd) to not prefer any particular label over the others. Guided by this, let us choose W and d that are label-ine iedee--nddeenden.. Substituting @ Φd + dΦ into q(:; Φ), we then obtain
ge(x; σ) = (") + s (sWifo(()++ + obe + W Of (æ; (e;6))8) + O(60²),
00
where é is the label index and óΦ (6W, 80, The lowest-order label-dependent correction to qe(2; Φ²) is given simply by (( + ôbg). In other words, in the lowest-order, the model only adjusts the final logits layer to adapt the pretrained embedding (x) to new task.
For simple softmax cross-entropy loss (belween predictions q(x)) and the groundtruth labels v), a single step of the gradient descent results in the following logits weight and bias updates:
sW,
2
Σ 1y== n
[0]] for (x).
8bg
? n (x,y)S
Σ
1y=k
1 )
(0.8)ES
where the I/|C| term results from normalization in the softmax operation. Here y is the learning rate, n is the total number of support-set samples, |C| is the number of classes and S the support set. In other words, we see that the label assignment imposed by oW and ab from the equation (7) effectively relies on computing dot-produee of (z) with "prototypes" c N-1 E(ewee foo y=)
6 Experiments
Most of our experiments were conducted using two standard benchmark problems using OMNIGLOT and TIEREDIMAGENET datasets. The generated weights for each task fg are composed of four comvolutional blocks and a single dense layer. Each of the convolutional blocks consist of a 3 x 3 convolutional layer, batch norm layer, ReLU activation and a 2 2 max-pooling layer. Preliminary experiments have showed that batch norm important to achieve the best accuracy. For OMNIgLOT we used filters for convolutional layers and 20-dim FC layer demonstrate how the network works on small problems, and for TEREDIMAGENET used 64 filters for convolutional and 40-dim for the FC laye: to show that the method works for large problems well. The models were trained in episodic fashion, where the examples for each training iteration are sampled from a given distribution of classes. The reported accuracy was calculated from 1024 random episodic evaluations from a separate test distribution, with each episode run 16 times with different combinations of input samples.
For the HT architecture, we tried to replicate the setup used in the original paper as closely as possible. We used a 4-layer convolutional network as feature extractor and a 2-layer convolutional model for computing activation features. For OMNIGLOT used a simplified 1-layer Transformer with 8 heads. In all our experiments, we trained the network on a single GPU for 4M steps with SGD with an exponential LR decay over 100 000 steps with a decay rate of 0.97. We noticed some stability issues when increasing the number of tasks and had to decrease the learning rate to compensate: for OMNIGLOT experiments, we used a learning rate 10 for up to tasks and 5 x 10 for lasks. For IIRREDIMAGENET, we used the same learning rate x 10-6 for training with any number of tasks T. We trained the CHT models with the inssemnnremettal objective 6 but evaluated them for both task-incremental and class-incremental scenarios.
used a 3-layer, 2-head Transformer and for TIereDMaGgENET we
II contrast with cross-entropy, do not need have the head layer dimension equal the number of predicted labels when using the Prototypical Loss.
8

---

Published in Transactions on Machine Learning Research (05/2024)
6.1 Learning from hini-batches
We first consider a case where every task includes the same set of classes. Specifically, we compared the following three models using set of four 5-way 1-shot support set batches ( ..... S(4) that consist of the same set of classes from TIBREDIMAGENET,
((a) = (s(()) + 8(2) + 5(3) + 5(4) ((01))0)) 00).
((0) (s 8(4) a (s())) + S 0u)),
((0) = (S(4), (s a (S (S (o)))),
where + operation denotes a simple concatenation of different support set batches. For this experiment, we used the cross-entropy loss (since the label set was the same for all ((i) and each support set batch S6 contained a single example per class. We observed that the test accuracies for gla), g(6) and (fc) were equal to 67.9%, 68.0% and 68.3% respectively, all within the statistical error range (+0.4%). At the same time, HT trained with just ((1) or (()) + +(2) (with or 2 samples per class respectively) performed significantly worse, reaching the test accuracies of 56.2% and 62.9% respectively. This demonstrates that the proposed mechanism of updating generated CNN weights using information from multiple support set batches can achieve performance comparable to processing all samples in a single pass with HT.
6.2 Learning from tasks within a single domain
Next, consider scenario where the tasks consist of classes drawn from single overall distribution. We present the results of two models: one trained on 20-way, 1-shot tasks with classes sampled from OMNIgLOT dataset, and anther trained on 5-way, 5-shot tasks with classes sampled from TIEREDIMAGENNT dataset.
We compare the performance of CHT to two baseline models. The first is a CONSTANT PROTONET (CoNsTPN), which represents a vanilla Prototypical Network, as described in [Snell et al. (2017). In this approach, universal fixed CNN network is trained on episodes from Cirain- This constant network can be applied to every task separately by projecting the support set prototypes for that task and computing the prediction with respect to these prototypes. Strietly speaking, this is not a continual learning method, since it treats every task independently and has no memory of previous tasks. For the best results on this basclinc, we had to increase the number of classes by a factor 5 during training (e.g. for 20-way OMNIGLOT evaluation we have trained with 100-way problems).
The second baseline we used specifically for the inssemnnreeetall learning is a MERGED HYPERTRRANS- FORMER (MERGEDHT). where combine all the tasks and train a single original HT instance as single task. This method does not solve a continual learning problem, since it has the information about all the tasks from the beginning, but it produces solution for every class and task that we can still be compared to the weights generated by the CHT.
Each trained model is applied to both task-incremente (Figure 3 and class incrementt (Figure settings. To understand the effect of continual learning with multiple tasks, each column represents a separate run of the CHT trained on T = 3, 4 or tasks in total (for training higher T, see the results in the Appendix). To demonstrate the recurence of the method, regardless of how many tasks it was trained on. Each plot shows 5 curves corresponding to the CHT, split into two groups: bullet marker (o) for tasks that the model was trained for and diamond marker (o) for extrapolation to more lasks.
extended the mumber of tasks to 5 for the evaluation
Task-incremental learning. We start by analysing the insk--ncremenntll learning results. For the Om- NIGLOT dataset, we saw no signs of catastrophic forgetting for the CHT. In fact, we observed a positive backward knowledge transfer, where the performance on past lasks improved as more weights were generated. For example, in most cases, the performance of 9, (green markers) was higher than 9, (orange markers), and 0g was higher than both 02 and tp- Additionally, as the number of tasks increased, the overall performance of the CHT also increased, with the model trained on T tasks performing better than the one trained
T = 2 tasks.
9

---

Published in Transactions on Machine Learning Research (05/2024)
T 2 tasks
T = tasks
T 4 tasks
T = 5 tasks
OMNIGLOT, 8 channels
0.88
0.87
0.86
0.85
0.84
Accuracy
2
3
0
2
3
0
2
3
TIERED IMAG ET, 64 channels
0.74
0.72
0.70
0.68
0
2
3
4
0
1
2
3
4
2
0
2
3
Task Name
0
0
22
03
O4
ConstPN
Figure 3: Task-incremental lcarning on OMNIGLOT and TIEREDIMAGENET. Each column represents a different CHT trained with a lotal of T = 2, 3, or 5 lasks. The tasks marked with a bullet symbol (+) correspond to the terms in the objective function 3 that are being minimized. The lines marked with the diamond symbol (o) show the extrapolation of the trained CHT to a larger number tasks. The confidence intervals do not exceed 0.5%.
T tasks
T= tasks
T= 4 tasks
T = 5 tasks
OMNIGLOT, channels
0.90
0.85
0.80
0.75
0.70
0.65
Accuracy
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
TIEREDIMAGENET. 64 channels
0.75
0.70
0.65
0.60
0
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
Task Name
0
01
02
03
O4
ConstPN
MergedHT
Figure Class-incremental learning on OMNIGLOT and TIEREDIMAGENE.. Fach column represents a different CHT trained with total of T 2, 3, or tasks. The tasks marked with bullet symbol (o) correspond to the terms in the objective function (3) that are being minimized. The lines marked with the diamond symbol (o) show the extrapolation of the trained CHT to a larger number of tasks. The confidence intervals do not exceed 0.5%.
10

---

Published in Transactions on Machine Learning Research (05/2024)
Table 1: Performance comparison of different methods for learning up to tasks on OMNIGLOT dataset.
Method
Task name
0
0-1
0-2
0-3
0-4
Pretraining
38.4
21.1
8.3
4.8
3.9
MAML++
81.4
58.2
33.5
24.4
19.8
EWC
33.4
20.4
9.0
4.7
3.7
Continual HT (ours)
87.2
80.0
75.1
76.8
69.3
For the TIBREDIMAGENET dataset, the results were better than the CONsTPN baseline, but the positive backward knowledge effect effect was not pronounced as it was for the OMNIGLOT dataset. The perfor- mance for every training lask remained roughly the same for all generated weights, indicating that the model did not suffer from catastrophic forgetting.
Overall, the CHT consistently outperformed the CONSTPN baseline, particularly when applied to the same or lower number of tasks it was trained on. Although the accuracy of the CHT did decrease slightly when it was applicd to more tasks than was trained on, this decrease was not significant. In fact, even when CHT was trained on only T = 3 (asks, generating weights for one of two additional tasks still resulted in better performance than the CoNSTPN baseline.
Class-incremental learning. In the class-incremental learning setting, the task name is given by two numbers indicating the range of tasks we used for evaluation (e-g. task name 0-3 corresponds to four tasks from 0 to 3). The black constant dashed line is the baseline performance of the CoNSTPN, which uses fixed embedding and does not differentiate between tasks. The starred blue markers represent separate run of the HT for a particular configuration of merged tasks.
As one can see in the Figure 4 the accuracy of all the models decreased as more tasks were included in the prediction. This was expected because the size of the generated CNN did not change, but the number of classes that needs to be predicted was increasing. For OMNIGLOT dataset we again saw the positive backwards transfer taking place, with CHT models trained more tasks T performing better overall. For comparable. This demonstrates the preemptive
given model trained on fixed T, the performance property of the CHT, where models trained for a certain number of tasks can still be run for any smaller number of tasks with similar performance.
When comparing the results to the baselines, the CHT had better results than the CONSTPN up to the number of tasks T was trained for, and the extrapolation results improved as T increases. Interestingly, for the case of T 5 the CHT was able to outperform even the MERGEDH'T baseline for the OMNIGLOT, even though the MERGEDHT had access to information about all tasks from the beginning. This suggests that having more classes to classify makes the learning problem difficult for the original HT, the image embeddings may not be able to learn good embeddings. This is particularly evident in the TIEREDMMGGENE dataset, where the performance of the MERGEDHT is so low that it falls below 60%, even for the 0-1 task.
We have also compared the results of CHT with other, more traditional baselines from few-shot learning and continual learning literature. Specifically, we consider the following methods: Pretraining. Pretraining a network on full training set, then finc-tuning it on an input sequence of tasks from the test set.
MAML+ An example of a Few-Shot Learning mothod (Antoniou et al. 2019
EWC. An example of a Continual Learning method, run EWC (Kirkpatrick et al. 2017 pretrained network from above.
Table shows the results. Notice that MAML++ performed well on the first task, but not able to retain the knowledge for subsequent tasks, because of catastrophic forgetting. Pretraiing methods and EWC were not able to learn sufficiently well even the first task, because they were not able to learn from few-examples.
6.3 Learning from tasks across multiple domains
In the experiments described above, the support and query sets for each task were drawn from the same general distribution, and the image domain remained consistent across all tasks. If the tasks were drawn
11

---

Published in Transactions on Machine Learning Research (05/2024)
from different distributions and different image domains, would expect task-agnostic CONSTPN approach to suffer in accuracy because it would need to find a universal representation that works well across all image domains, In contrast, the CHT approach could adapt its sample representations differently for different detected image domains, leading to improved performance.
We verify this by creating a multi-domain episode generator that includes tasks from various image datasets: OMNIGLOT, CALTECH101, CALTECuBBRDS2011, CARS196, OXFORDFLOWERS 102 and STANFORDDoGS. We compared the accuracy of the CONSTPN and CHT on this generator using episodes containing two tasks with 5-way, 1-shot problems. The generated CNN model had 16 channels with 32 channels for the final layer. Other paramcters were the same those used in the TIBREDIMAGENET experiments. The CoNSTPN achieved the accuracy of 53% for task O, 52.8% for task 1 and 50.8% for combined tasks. The CHT achieved the accuracy of 56.2% for task 0, 55.2% for task 1 and 53.8% for combined tasks. The accuracy gap of nearly 3% between these two methods, which is larger than the gap observed the OMNIgLOT and TiEreDiMAGENeT experiments, suggests that the CHT better at adapting a multi-domain task distribution.
7 Discussion
While the computational cost of training the hypernetwork can be quite large, the training process is one- time investment on meta-training distribution of tasks. Once trained, the HyperTraansforerr can generate target CNN architectures for new sets of tasks in matter of seconds (e.g., 10 seconds as demonstrated in our experiments). This efficiency in generating new architectures is a key advantage when operating in dynamic task environments.
Since Hyper Transfommre requires a separate Transformer network in order to generate target CNN, whose architecture and optimization parameters can be considered arper-parameters of our method. However, in scenarios with novel dalasets, a practitioner may need to explore hyperparameter luning to optimize perfor- mance. A separate sensitivity analysis needs to be done to understand the role of various hyperparatemeterss of a generative Transformer network.
8 Conclusions
The proposed CONTINUAL TYPERRRRANEFORM model has several attractive features. As an efficient few-shot learner, can generate CNN weights on the fly with no training required, using only a small set of labeled examples. As a continual learner, it is able to update the weights with information from new tasks by iteratively passing them through HT. Empirically, we have shown that the learning occurs without catastrophic forgetting and may even result in positive backward transfer. By modifying the loss function from cross-entropy to the prototype loss, we defined a learning procedure that optimizes the location of the prototypes of all the classes of every task. A single trained CHT model can be used in both mask-incremental and class-incremental sccnarios.
9 Broader Impact Statement
This paper is focused on developing new methods for continual few-shot learning. We do not envision these methods being used for harmful purposes more so than other algorithms in a class of few-shot or continual learning.
The authors of this paper committed to making sure that the methods we develop are used with safety concerns in mind. We believe that continual few-shot lcarning has the potential to be a powerful tool for good, but it is important lo use it responsibly. While the technology itself does not directly facilitate harm to living beings or raise immediate safety concerns, its broader impact needs to be carefully considered in light of potential ethical, socictal, and environmental implications. We are committed to working with the research community to mitigate potential risks. We welcome any feedback from the reader regarding a potential misuse of the methods described in the paper that did not describe in this section.
12

---

Published in Transactions on Machine Learning Research (05/2024)
10 Acknowledgements
The authors would like to thank Nolan Miller, Gus Kristiansen, Jascha Sohl-Dickstein and Johannes Oswald for their valuable insights and feedback throughout the project.
References
Antreas Antoniou, Harrison Edwards, and Amos J. Storkey. How train your MAML. In 7th Interna- tional Conference Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
Antreas Antoniou, Massimiliano Patacchiola, Mateusz Ochal, and Amos Storkey. Defining benchmarks for continual few-shot learning. arXiv preprint arXi::2004.11 11 11967, 2020.
Magdalena Biesialska, Katarzyna Biesialska, and Marta Costa-Jussa. Continual lifelong learning natural language processing: A survey, arXiv preprint arXiv:201..08823 2020.
Kuilin Chen and Chi-Guhn Loc. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Regresenlations, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostie meta-learning for fast adaptation of deep networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1126 1135. PMLR, Aug 2017.
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the TEEE conference compuler vision and pottern recognition, pp. 4367-4375, 2018.
Gunshi Gupta, Karmesh Yadav, and Liam Paull. Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems,
2020.
David Ha, Andrew Dai, and Quoc V Lc. Hypernetworks. ar preprint arXiv:1609.09106 2016.
Yen-Chang Hsu, Yen-Chene Liu, Anita Ramasamy, and Zsolt Kira. Ro-evaluating continual learning scenar- ios: A calegorization and case for strong baselines. arXiv preprinl 0rx:1:1810.1124 2018.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Srabska-Barwinskie et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 3526, 2017.
Eugene Lee, Cheng-Haa Huang, and Chen-Y Yi Lee. Few-shot and continual learning with attentive indepen- dent mechanisms. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp.
2021.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Jaya Krishna Mandivarapu, Blake Camp, and Rolando Estrada. Self-Net: Lifelong learning via continual self-modeling. Frontiers in Artificial Intelligence, 3:19, 2020.
Pratik Mazumder, Pravendra Singh, and Piyush Rai. Fhw- lifelong learning. ar preprint
arXiv:2103. 00091, 2021.
Tsendsuren Munkhdalai and Hong Yu. Meta networks. In International Conference on Machine Learning, pp. 2554 2563. PMLR, 2017.
Alex Nichol and John Schulman. Reptile: scalable metalearning algorithm. arXiv preprint arXim:1803.00000 2(3):4, 2018.
13

---

Published in Transactions on Machine Learning Research (05/2024)
Boris N. Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. TADAM: task dependent adaptive metric for improved few-shot learning. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Lesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NearIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 719 729, 2018.
Ramakanth Pasumuru, Veselin Stoyanov, and Mohit Bansal, Continual few-shot learning for text classifica- tion. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5688 5702, 2021.
Juan-Manuel Perez-Rua, Xiatian Zhu, Timothy M Hospedales, and Tao Xiang. Incremental few-shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13846-13855, 2020.
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulié, Sebastian Ruder, Kyunghyun Cho, and Tryna Gurevych. Adapterhub: A framework for adapting transformers. arXio preprinl arXiv:2007.07779, 2020.
Mengye Ren, Renjie Liao, Ethan Fotaya, and Richard Zemel. Incremental few-shot learning with attention attractor networks. Advances in Neural Information Processing Systems, 32, 2019.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11 11910, 2018.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert. Soyer, James Kirkpatrick, Koray Kavukeuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1606v 04671, 2016.
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascamu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
Guangyuan Shi, Jiaxin Chen, Wenlong Zhang, Li-Ming Zhan, and Xiao-Ming Wu. Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima. Advances in Neural Information Pro- cessing Systems, 34:6747 6761, 2021.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning compare: Relation network for few-shot learning. In 2018 IEFE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 1199 1208. IEEE Computer Society, 2018. doi: 2.11099/VVP.2018...
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong, Few-shot class- incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12183 12192, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 5998 6008, 2017.
14

---

Published in Transactions on Machine Learning Research (05/2024)
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Daniel D. Lee, Masashi Sugiyama, Urrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 3630-3638, 2016.
Johannes Von Oswald, Christian Henning, João Sacramento, and Benjamin F Grewe. Continual learning with hypernetworks. arXiv preprint 1rXi: 1906.0095, 2019.
Liyuan Wang, Qian Li, Yi Zhong, and Jun Zhu. Few-shot continual learning: brain-inspired approach. arXiv preprint ar Xix:2110 09034, 2021a.
Xin Wang, Yue Liu, Jiapei Fan, Weigao Wen, Hui Xue, and Wenwu Zhu. Continual few-shot learning with transformer adaptation and knowledge regularization. In Proceedings of the ACM Web Conference, volume 2023, 2023.
Yu Wang, Nicholas J Bryan, Mark Cartwright, Juan Pablo Bello, and Justin Salamon. Few-shot continual learning for audio classification. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 321-325. TEFE, 2021b.
Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. arXiv preprint arXie:2002.0677 06715, 2020.
Li Yin, Juan M Perez-Rua, and Kevin J Liang. Sylph: A hypernetwork framework for incremental few- shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1035-90455 2022.
Friodemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In PMTR, 2017.
International Conference on Machine Learning, pp.
Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu, Few-shot incremental learning with continually evolved classifiers. Proceedings of the TEFE/CV /CF Conference on Computer Vision and Pattern Recognition, pp. 12455-12464, 2021.
Andrey Zhmoginov, Mark Sandler, and Max Vladymyrov. Hypertransformer: Model generation for super- vised and semi-supervised few-shot learning. arXiv preprint arXiv:2201.041 04188 2022.
15

---

Published in Transactions on Machine Learning Research (05/2024)
0.8
Accuracy
0.6
0.87
0.4
Accuracy
0.2
0.9
Task name
0.8
6 +
8,
0, 6,
ConstPN
0.7
Figure 5: The accuracy of the HT model trained for T 5 using the cross-entropy loss. The curacy of the first weight On is high and is better than the accuracy of the CONSTPN model's em- beddings. However, when more tasks are added, the accuracy drops dramatically due to collisions between the same classes for different lasks the cross-entropy loss.
0.6
0-2
0-4
Task name
min eq.
5
0.
02
02
0g
0.
min eq.
6
8
9
82
83
8.
Figure 6: CHT trained using iask-incremental ob-
jective (5 vs. class-incremental objective
A
Additional experiments and figures
A.1 Learning with cross-entropy loss
Figure 5 shows the results of an attempt to do learn multiple tasks using HT with cross-entropy loss. Since the size of the last layer's embedding is not increased, the model can only predict the class labels within the task and not the task themselves, which corresponds to the domain-incremenn incremental learning setup. Additionally, the same class from different tasks are mapped to the same location in the embedding space, leading to collisions when more tasks are added. This is why the accuracy drops siguificantly as the number of tasks increases. On the other hand, CoNsTPN model is more flexible because the prototypes for each task are computed from the support of that task and do not have to be fixed a one-hot vector as in the cross-cntropy loss.
A.2 Training using task-incremental and class-incremental objectives
Figure B compares the accuracy of two different models trained with task-incremental (using equation and class- incremental (using equation ( objectives. The performance of both models on task-incrementar problems are similar, while the model trained with the class-incremental objective performs better on class- incremental problems.
A.3 Analysis of prototypical embeddings using UMAP
To better understand the quality of the learned prototypes, we conducted an experiment on a TIEREDIMA- GENET 5-way, 5-shot problem. We selected a random episode of 5 tasks and ran them through the model, producing weights & to 04 along with the prototypes for each class of every task. We then computed the logits of the query sets, which consisted of 20 samples per class for cach task. The resulting 40-dim em- beddings for the prototypes and query sets were concatenated and projected onto 2D space using UMAP (Figure 7. Note that the prototypes from the earlier tasks remain unchanged for the later task, but their 2D UMAP projections are different, because UMAP is non-parametric method and it must be re-run for every new 0g. We tried our best to align the embedding using the Procrustes aligument method.
The plot shows that the embeddings of the tasks are well separated in the logits space, which helps explain why the model performs well for both task- and iass-incremental learning. Normalizing the softmax over the classes within the same tasks or across all tasks made little difference when the tasks are so far away from each heher. On the right of Figure we show the projection of the CoNsTPN embedding of the same
16

---

Published in Transactions on Machine Learning Research (05/2024)
0
02
0g
O4
CoNSTPN
Figure 7: Left plots: the UMAP projection of the CHT prototypes and the query set embeddings for different generated weights, where the points colored based the task information. The query set points are connected with their corresponding prototypes using dashed lines. Right plot: UMAP projection of the CoNsTPN embedding for 25 different classes from TIBREDIMAGENET. Embeddings are aligned using Procrustes alignment.
0o
1
02
O3
04
Figure 8: The UMAP projections of 20-dimensional embeddings of the prototypes and query set for different weights obtained from incremental HT training. The query set points are connected to their corresponding prototypes using dashed lines. In the top plot, the points are colored according to their class information, while in the bottom plot they are colored according to their task information. Embeddings are aligned using Procrustes alignment.
25 classos. The CONSTPN model does not make a distinction between tasks and treats cach class scparately. The fact that clusters emerge has lo do purely with the semantics of the chosen clusters and the way the CoNSTPN model groups them. This also helps to explain why the CHT model performs better than the CONSTPN, as it separates the tasks before separating the classes within each task.
The UMAP embedding for the OMNIGLOT dataset using ProtoNet (Figure g appears to be different from similar embedding projection of MIRREDIMAGENET dataset. In particular, the embeddings from different tasks seem to overlap, while the TIereDIMAgENdr embedding they are separated. This may due to the fact that the classes in the OMNIGLOT dataset are more closely connected than those the TieREDIMAGENeT dataset. Interestingly, despite the overlap between the classes from different tasks, the final accuracy is still high and only slightly degrades more tasks added.
A.4 Learning with more tasks
Our analysis primarily focused on the performance of the CHT on up to tasks. However, as shown Figure the CHT model is capable of handling much larger number of tasks T. Similar to the results in
17

---

Published in Transactions on Machine Learning Research (05/2024)
OMNICLOT, channel
OMNIGLOT, 32 channel
0.85
0.8
Number taaks
0.50
Number tasks 10
Aours 327
Accurazy
Numbbr takks
0.86
0.80
0.0
60
0-16
0-10
0-10
0
09
Task namo
Task
Figure 9: OMNIGLOT with 8 or 32 channels trained with a different number of tasks T.
85
0.570
0.
8%9
Accuracy
Accuracy
0.865
0.860
0-1
0-3
044
03
06
0-7
090
0.10
0-11
0-12
0-13
0-14
0-16
0-16
0-17
0-18
0-19
9 10
11
12
13
14
15
Task name
Task name
Figure 10: OMNIGLOT with 8 channels trained for 7 = 20 tasks. Here show the final weight 019 generated
along with an intermediate 0g for task-incremental (left) and class- caassincremmntal (right) learning.
the main text, the nearly overlapping curves in the graph indicate that the model trained for tasks can maintain the same level of accuracy when applied to a larger number of tasks.
Figure 10 shows 8-channel OMNIgLOT evaluated on task-incremental and class-incremental objectives.
A.5 Continual Transrranfformer vs MergedHT for tieredImageNett
Figure 11 shows a zoomed out view of the results presented in Figure 4h. illustrates the significant difference performance between the MERGEDHT and the CHT models.
A.6 Additional figures for Omniglot task-incremental and class-incremental learning
Figures 12
and 15 show additional experiments with the OMNIGLOT dataset using different number
of channels the CNN.
T = 2 tasks
T= 3 tasks
T 4 tasks
T = 5 tasks
TIEREDIMAGENET, 64 channels
Accuracy
0.7
0.6
0.5
0.4
0.3
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
0-2
0-3
0-4
0-1
0-2
0-3
0-4
Task Name
0
02
0
04
ConstPN
MergedHT
Figure 11: Zoomed out view of Figure
that the results of the MERGEDHT is visible.
18

---

Published in Transactions on Machine Learning Research (05/2024)
T = tasks
T 3 tasks
T= 4 tasks
T 5 tasks
Task-incremental learning
0.675
0.675
0.675
0.675
0.650
0.650
0.650
0.650
0.625
0.625
0.625
0.625
Accuracy
0
2
3
0
2
3
2
3
4
1
2
4
Class-increment learning
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
Task Name
0
02
03
O4
Figure 12: Task-incremental and class-incremental learning the OMNIGLOT dataset with 4-channels volutions.
tasks
lasks
T= 4 Lasks
lasks
Task-incremental learning
0.80
0.80
0.80
0.80
0.78
0.78
Accuracy
2
2
3
4
0
2
3
4
ilass-incrementll learning
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
Task Name
O.
0
O2
0
Figure 13: Task-incremmnnal and dlss-incremmntal learning on the OMNIGLOT dataset with 6-channels con- volutions.
19

---

Published in Transactions on Machine Learning Research (05/2024)
T 2 tasks
T= 3 tasks
T= 4 tasks
T = 5 tasks
insemmnnremennal learning
0.95
0.95
0.95
0.95
0.94
0.94
0.94
0.94
0.93
0.93
0.93
0.93
Accuracy
2
3
0
2
3
4
Class-incrementee learning
0.95
0.95
0.95
0.95
0.90
0.90
0.90
0.90
0.85
0.85
0.85
0.85
0.80
0.80
0.80
0.80
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
0
0-1
0-2
0-3
0-4
Task Name
0
0₂
θ
O4
Figure 14: Task-incremental and class-incrementee learning on the OMNIGLOT dataset with 16-channels convolutions.
T = tasks
T = 3 tasks
T 4 tasks
T 5 tasks
Task-incremental learning
0.960
0.960
0.960
0.960
0.955
0.955
0.955
0.955
0.950
0.950
0.950
0.950
Accuracy
3
2
Mlass-incremental learning
0.95
0.95
0.95
0.95
0.90
0.90
0.85
0.85
0.85
0.85
0-1
0-2
0-3
0-4
0-1
0-2
0-3
0-1
0-2
0-4
0
0-4
Task Name
O
0
02
0
Figure 15: Task-incremental and class-incremental learning on the OMNIGLOT dataset. with 32-channels convolutions.
20


Reviews:
[{"review_id": "I1cHlxqqGf", "reviewer": "Reviewer_sN8z", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I have broad issues with this paper that I hope the authors can either address via clarification or make changes to the paper to address\n\n**Past work not properly contextualized through baselines / discussion**\n\n[1] arguable does exactly what your paper claims to be doing but very little comparison to it is made in the paper. \nThe paper does mention other continual learning approaches like memory buffers for re-training and weight regularization methods but none of these are compared against. For example, [1], which is a very similar paper, compares to online EWC (elastic weight consolidation), which is a reasonable baseline to compare against.\n\nAlso, another point to note is that it is unclear that the the method the paper proposes actually does better than the simple baseline they implemented (ConstPN).  Figure 3 has no error bars around the plots. However, the deltas in performance are on the order of  <= 2%. Thus,  error bars would be great in order to gauge whether their results are actually significant.\n\n**Limited discussion of weaknesses**\n\nThe method has several weaknesses that are not discussed including :\n- complexity approach given the need to train a hyper-network\n- the introduction of extra hyper-parameters due to the hyper-network\n- memory and compute cost of training a hyper-network\n\n**Implications of section 6.1**\n\nI think I took away a different message from the results of section 6.1 than the authors present as reason the nature of their results.\nFirst, note that all 3 types of models (a), (b), (c), produced near identical results. This to me indicates that the model is unable to perform positive forward transfer. Next, the authors say training on only S(1) or S(1) + S(2) produces poor results. This should not be surprising though since this is strictly less data than (a), (b) and (c) approaches have seen.  So I don't think this result is actually a testament to their method.\n\n**Unsubstantiated claim**\n\nIn the last but 3rd paragraph of the introduction, you state that the models you learn with CHT *do not suffer from catastrophic forgetting*. This is a bold claim to make given that a) you only evaluate on task horizons up to 5  b)  Even for the Ominglot experiments in Fig 3, one can see if that if you generate $\\theta_t$ beyond the number of training timesteps, $\\theta_t$ performs worse than $\\theta_i$ on task $i$, meaning that forgetting does occur when you extend the model past the training horizon\n\n\n[1] CONTINUAL LEARNING WITH HYPERNETWORKS -- https://arxiv.org/pdf/1906.00695.pdf", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an approach to continual few-shot learning based on a recurrent hyper-network. The hyper-network is trained to predict the weights of a CNN, given the previously set of predicted task weights and the support set of the new task. At inference time, such an approach would allow for gradient-free continual learning.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the issues I have discussed above", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "76BGTUvluj", "reviewer": "Reviewer_nY3Y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Overall, this is an ok paper, with a modest technical contribution and ok empirical justification. The proposed model is an adaptation of an existing one that works well with few-shot learning, that is HyperTransformers. Empirical validation shows that the continual version of it can also handle streaming settings.\n- The related work is broadly covered with respect to neighbor fields, including few-shot learning, continual learning, etc.\n- In that sense, this paper also extends existing works towards a novel data setting, where not only the model is exposed to limited data (aka few-shot learning) but is done in a continual way also.\n\nWeaknesses\n- The paper is not as exciting, in that it seems an existing method and an existing data setting are rather straightforwardly extended and results are decent but unsurprising. I understand that this is a comment that is hard to argue for or against, my point is that at the moment there seems to be no strong conclusion or insight from the work. Indeed, adapting an existing model towards a specific new (although not unheard of) direction gives the expected results. In that sense, the novelty is on the low side.\n- I think the comparisons, and general positioning is lacking, which is also evident in that the paper makes only 'internal' comparisons but does not compare much against other published baselines. For instance, I know that T. Tuytelaars from KU Leuven has worked extensively on continual learning as well as few-shot learning, including also a survey on the subject matter (and I am sure there must be more baselines on such a broad topic). This is missing at the moment, and I would advise looking at the related literature before having a more extensive review\nhttps://scholar.google.be/citations?hl=en&user=EuFF9kUAAAAJ&view_op=list_works&sortby=pubdate\nhttps://scholar.google.be/citations?view_op=view_citation&hl=en&user=EuFF9kUAAAAJ&citation_for_view=EuFF9kUAAAAJ:isU91gLudPYC\n\nSome more works with simple Google search\nhttps://aclanthology.org/2021.emnlp-main.460/\nhttps://dl.acm.org/doi/abs/10.1145/3543507.3583262\nhttps://github.com/huang50213/AIM-Fewshot-Continual\n\n- I find the text clear but the overall composition and description of the method lacking sufficient detail. For instance, it is not clear what is the definition of tasks here, as it is not explained really, not even in the experiments section. One could think of as tasks to be new types of classes (but is this then a 'new task' or simply extending the class set)? A more natural definition would be having classification, localization, segmentation, detection etc as different tasks. Either way, right now it is not easy to understand what exactly the method does and whether the empirical validation is sufficient. More details in random order\n-- Why is positive backward transfer working? Is there any theoretical or intuituve justification?\n-- In general, the whole claim is rather ad hoc, which is I guess anyway expected, as it is very hard to prove theoretically that an online algorithm would generalize. However, I feel a bit uneasy confirming that this algorithm would necessarily work well on a new dataset or a new set of tasks.\n-- The CNN weights are generated, thus the CNN has no training itself. Rather, the training is on the transformer that generates the CNN parameters on the fly. If my understanding is correct, does one really need a batch norm layer which is mostly for training? I suppose one would still need it for practical purposes, or because even 'indirect' training would require some form of gradient normalization, however, I would expect some clarity there.\n-- On a similar matter, if the classifier is determined dynamically as in equations 4-6 with prototytpes, is there potentially a problem of scale, when adding many new classes, thus creating gradients of much larger magnitudes?\n-- What exactly is the point of section 5? Does it give any new insight from MAML literature or is it more of a 'hey look what i found out' highlight?\n-- How important is the order with which the classes and tasks are presented. Especially when not having a recurrent setting with shared weights, wouldn't this affect results significantly? I would expect at least an ablation study here.\n\nIn general, I think that this is a very hard task that the paper is addressing and I want to express my appreciation for the difficulty.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work focuses on continual few-shot learning, whereby a model learns over a stream of data and tasks while having access to few data points at the time and no access to past data. An adaptation of an existing model, HyperTransformers, is proposed called Continual HyperTransformers. The adaptation is straightforward, conditioning the model on past tasks, specifically the generated weights from past tasks. The paper claims the model exhibits limited forgetting and shows evidence of the opposite, called 'positive backward transfer', that is that the model gets better with new tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See above.", "broader_impact_concerns": "Ok.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dI6iOuy7qx", "reviewer": "Reviewer_frdT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n+ Overall, the various scenarios of continual few-shot learning that the paper covers, related work and all experimental setups are presented clearly.\n+ Extensive experiments showcase the validity of the proposed method, CHT.\n\nWeaknesses\n- The authors should describe the architecture of CHT in more detail. Figure 2 alone is not sufficient for explaining the model architecture.\n\nThe well-designed experiments explore various aspects of CHT in the target problem, continual few-shot learning, which clearly shows the effectiveness of the method. It seems that this work is quite incremental because it adapts the existing model, HT, to the continual learning and does not introduce any new components in the training objective; it just uses the existing prototypical loss. However, I don't believe that this should not be a reason for rejection by itself, according to the acceptance criteria of the journal.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a variant of HyperTransformer (HT) originally developed for few-show learning, Continual HyperTransformer  (CHT) for the task of continual few-show learning. By feeding previously generated weights into CHT, the generated weights contains information from a new task while retaining the knowledge about tasks have already been seen. With the prototypical loss, CHT achieves competitive results on two standard benchmarks, Omniglot and tieredImageNet, in several continual few-shot learning scenarios.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please describe the architecture of CHT in more detail.\n- Please add more baselines, if possible. Comparing with only a single baseline is not enough for verifying the superiority of the proposed method (MergedHT is used only for class-incremental learning on Omniglot).", "broader_impact_concerns": "The authors adequately addressed the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The proposed CHT method is a straightforward extension of the previously published HyperTransformer approach and therefore has relatively low novelty. It is important to note, however, that novelty is not a criterion for acceptance to TMLR. Comparison to baselines was also raised as an issue by reviewers. Due to the relative novelty of the continual few-shot learning setting, there are not clear established baselines in the literature to compare against. During the response period, experiments comparing to both few-shot and continual learning approaches were added and shown to perform much worse than the proposed CHT model.

Overall, the claims of this paper are somewhat modest, but are supported sufficiently by experiments and should be of interest to the community. Therefore, I recommend acceptance with the following minor revisions:
- Sufficient details regarding the HyperTransformer (as background material) and the Continual HyperTransformer should be added to allow for others to reproduce the results.
- Error bars should be added to the plots, especially Figures 3 and 4.

Additionally, the authors are encouraged to include the following modifications, which would improve the quality of the paper but are not critical to securing acceptance:
- Ablations showing the effects of (a) batch normalization and (b) the order of classes and tasks.
- Broader comparison to baseline continual learning methods, such as Online EWC (Schwarz et al. 2018) and “Continual Learning with Hypernetworks” (von Oswald et al. 2020). Although such approaches may not be specifically designed for the few-shot continual learning setting, it is still valuable to benchmark against them to demonstrate the challenges of the few-shot continual setting and the degree to which CHT is able to address the gap.
Decision: Accept (poster)